7;hdZddlmZddlmZddlmZddlZddlmZddl m Z m Z edgd Z d Z dd e e efd Zdd ZddedefdZy)zToken-related utilities) namedtuple)StringIO) iskeywordN) TokenInfo)ListOptionalToken)tokentextstartendlinec#|K tj|D]}|y#tj$rYywxYww)z'wrap generate_tkens to catch EOF errorsN)tokenizegenerate_tokens TokenError)readliner s h/workspace/moltex-experiments/.pixi/envs/default/lib/python3.12/site-packages/IPython/utils/tokenutil.pyrrs<--h7EK8   s<#<9<9<extra_errors_to_catchc# Kgd}|t|tsJ||xsgz}g} tj|D]}|j ||y#tj $r t fd|Drh|r6|djd|djdf}|}|dj}ndx}}d}tjtjd|||nYd yd wwxYww)N)zunterminated string literalzinvalid non-printable characterz!after line continuation characterc3@K|]}|jdvyw)rN)args).0errorexcs r z/generate_tokens_catch_errors..,sAu #sr)r) isinstancelistrrappendranyr r rr ERRORTOKEN) rrdefault_errors_to_catcherrors_to_catchtokensr r r rrs @rgenerate_tokens_catch_errorsr(s ! (J7Ld,SS S-1F1L"MO F--h7E MM% K8     AA Ar ((+VBZ^^A->>bz$$$$X%8%8"eS$O O  P s/$C5/AC5C2+A=C-(C5-C22C5cd}|jd}|D]2}|t|z}|jds|dz }||kDr||fS|}4d}||fS)aReturn the line in a cell at a given cursor position Used for calling line-based APIs that don't support multi-line input, yet. Parameters ---------- cell : str multiline block of text cursor_pos : integer the cursor position Returns ------- (line, offset): (string, integer) The line with the current cursor, and the character offset of the start of the line. rT rr) splitlineslenendswith)cell cursor_posoffsetlinesr next_offsets rline_at_cursorr3:sv"F OOD !Es4y( }}T" 1 K  #  &> &>r.r/c.g}g}g}ddi}tt|jD]}t|}|j\}} |j \} } | dz|vrN|j jd} t| |dzD]!\} }| |vs || dz t|z|| <#||}| dk(r|dzn|}|| z|k\rnD|jtjk(rt|jsj|rL|rJ|djtjk(r*|djdk(r|dd|j|d<n|j!|jn|jtjk(rj|jdk(r|r|j#d|jdk(r|r|j!|dn"|jdk(r|r|j#d|j!||| | z|kDsn|r|dS|r|dSy ) avGet the token at a given cursor Used for introspection. Function calls are prioritized, so the token for the callable will be returned if the cursor is anywhere inside the call. Parameters ---------- cell : str A block of Python code cursor_pos : int The location of the cursor in the block where the token should be found rrTr.=()r)rrrr r r rr+ enumerater,r rNAMErr OPr"pop)r.r/namesr' call_namesoffsetstuptok start_line start_colend_lineend_colr1linenorr0boundarys rtoken_at_cursorrI\sEFJ!fGx~667Sk!$  IGG' a I  )  99 %i.AF2J$4$4 $Cr [^H^',Ry#((;b  SXX& YY(++ %xx35 " xx35!!%),SZr" c 8 w & 3 S8V"~ Ryr4)N)r)__doc__ collectionsriorkeywordrrrtypingrrr rstrr(r3intrIr4rrRsh #! 7EF<@%-d3i%8>DD#D3Dr4