BXQc @svdZddddddddd d d d d ddddddddddddddddddd g Zd!d"lZd!d#lmZd!d"lZd!d"lZd!d"lZd!d"lZd!d"l Z d!d"l Z d!d"l Z d!d"l Z d!d"l Z d!d"lZd!d"lZd!d"lZd$Zd%Zd&Zejd'Zead(Zd)Zd*d+Zdfd,YZejd-Zd.Zd/Zed0Z d1Z!d2Z"d3Z#d4Z$d5Z%d6fd7YZ&de'fd8YZ(de(fd9YZ)de(fd:YZ*de(fd;YZ+d e'fd<YZ,dej-ej.fd=YZ/d>fd?YZ0d e0ejfd@YZ1dAe0fdBYZ2de2ej3fdCYZ4dDe2fdEYZ5dFe5e j6fdGYZ7dHe5e j8fdIYZ9e:e4ej;e<ee j=e j e j>dJZ?e:e4ej;e<ee j=e j e j>dKZ@dLZAdMZBdNZCe:e:e4ej;e<eee j=e j e j>dO ZDdfdPYZEdQZFdfdRYZGdeGfdSYZHdeHfdTYZIdeHfdUYZJd eHfdVYZKd eHfdWYZLd fdXYZMdYZNdeGfdZYZOdeOfd[YZPdeOfd\YZQdeOfd]YZRdeHfd^YZSd eSfd_YZTdeIfd`YZUdeIfdaYZVdeIfdbYZWdeSfdcYZXddZYdfdeYZZd"S(fsHTML form handling for web clients. HTML form handling for web clients: useful for parsing HTML forms, filling them in and returning the completed forms to the server. This code developed from a port of Gisle Aas' Perl module HTML::Form, from the libwww-perl library, but the interface is not the same. The most useful docstring is the one for HTMLForm. RFC 1866: HTML 2.0 RFC 1867: Form-based File Upload in HTML RFC 2388: Returning Values from Forms: multipart/form-data HTML 3.2 Specification, W3C Recommendation 14 January 1997 (for ISINDEX) HTML 4.01 Specification, W3C Recommendation 24 December 1999 Copyright 2002-2007 John J. Lee Copyright 2005 Gary Poster Copyright 2005 Zope Corporation Copyright 1998-2000 Gisle Aas. This code is free software; you can redistribute it and/or modify it under the terms of the BSD or ZPL 2.1 licenses (see the file COPYING.txt included with the distribution). tAmbiguityErrortCheckboxControltControltControlNotFoundErrort FileControlt FormParsertHTMLFormt HiddenControlt IgnoreControlt ImageControltIsindexControltItemtItemCountErrortItemNotFoundErrortLabelt ListControlt LocateErrortMissingt ParseErrort ParseFilet ParseFileExt ParseResponsetParseResponseExtPasswordControlt RadioControlt ScalarControlt SelectControltSubmitButtonControlt SubmitControlt TextControltTextareaControltXHTMLCompatibleFormParseriN(tStringIOs0.2.11islatin-1smechanize.formscOsLtr dStjdd}d|}|f|}tj|||dS(Niis%%s %s(tOPTIMIZATION_HACKtinspecttstackt_loggertdebug(tmsgtargstkwdst caller_namet extended_msgt extended_args((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyR%[s   cCsItatjtjtjtj}|jtjtj |dS(N( tFalseR!R$tsetLeveltloggingtDEBUGt StreamHandlertsyststdoutt addHandler(thandler((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyt_show_debug_messagesds icCstj|tdd|dS(Nt stackleveli(twarningstwarntDeprecationWarning(tmessaget stack_offset((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyt deprecationmscBseZRS((t__name__t __module__(((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRqss\s+cCstjd|jS(Nt (t _compress_retsubtstrip(ttext((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyt compress_texttscCstjdd|S(Ns(?:(?|dksd|kr|S||d}tjd||S(Nt&cSs|j}|ddkr0t|dd!|S|j|}|dk rt|tdkry|j|}Wqtk r|}qXqn|}|S(Nit#iit(tgrouptunescape_charreftgettNonettypetencodet UnicodeError(tmatchtentitiestencodingtenttrepl((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pytreplace_entities~s   s&#?[A-Za-z0-9]+?;(RMRERA(tdataRRRSRV((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pytunescapezscCs|d}}|jdr0|dd}}ntt||}|dkrU|Sy|j|}Wntk rd|}nX|SdS(Ni txiis&#%s;(t startswithtunichrtintRMRORP(RWRStnametbasetucRU((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRKs   cCsddl}ddlm}i}y |jWntk ri}x|jjD]a\}}||d}|jdr|jdrt |dd!d}n||d| w = MimeWriter(f) ...call w.addheader(key, value) 0 or more times... followed by either: f = w.startbody(content_type) ...call f.write(data) for body data... or: w.startmultipartbody(subtype) for each part: subwriter = w.nextpart() ...use the subwriter's methods to create the subpart... w.lastpart() The subwriter is another MimeWriter instance, and should be treated in the same way as the toplevel MimeWriter. This way, writing recursive body parts is easy. Warning: don't forget to call lastpart()! XXX There should be more state so calls made in the wrong order are detected. Some special cases: - startbody() just returns the file passed to the constructor; but don't use this knowledge, as it may be changed. - startmultipartbody() actually returns a file as well; this can be used to write the initial 'if you can read this your mailer is not MIME-aware' message. - If you call flushheaders(), the headers accumulated so far are written out (and forgotten); this is useful if you don't need a body part at all, e.g. for a subpart of type message/rfc822 that's (mis)used to store some header-like information. - Passing a keyword argument 'prefix=' to addheader(), start*body() affects where the header is inserted; 0 means append at the end, 1 means insert at the start; default is append for addheader(), but insert for start*body(), which use it to determine where the Content-type header goes. cCs1||_||_g|_g|_t|_dS(N(t _http_hdrst_fpt_headerst _boundaryRot _first_part(tselftfpt http_hdrs((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyt__init__s     icCs|jd}x|r-|d r-|d=qWx|rL|d rL|d=q1W|rdj|}|jj|j|fnx5tdt|D]}d||j||t__doc__RMRRRRRRR(((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyR{sD      cBseZRS((R=R>(((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRZscBseZRS((R=R>(((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyR[scBseZRS((R=R>(((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyR\scBseZRS((R=R>(((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyR ]scBseZRS((R=R>(((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyR _scBseZdZdZRS(cOstj|||dS(N(t ExceptionR(RR'R(((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRgscCs tj|S(N(Rt__str__(R((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRjs(R=R>RR(((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRds t_AbstractFormParsercBseZdZdedZdZdZdZdZ dZ dZ dZ d Z d Zd Zd Zd ZdZdZdZdZdZdZdZdZdZdZdZdZdZdZRS(s:forms attribute contains HTMLForm instances on completion.cCs|dkrt}n||_||_d|_g|_g|_d|_d|_d|_ d|_ d|_ d|_ d|_ |jg|j|jd|_|_ dS(Ni(RMRkt _entitydefst _encodingR^tformstlabelst_current_labelt _current_formt_selectt _optgroupt_optiont _textareat _global_formt start_formtend_form(RRfRS((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRqs"                cCsItd|x5|D]-\}}|dkr|j||_qqWdS(Ns%sthref(R%tunescape_attr_if_requiredR^(RtattrsRR((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pytdo_bases  cCsItd|jdk r&|jn|j|jk rE|jndS(NRI(R%RRMt end_labelRRR(R((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pytend_bodys   c Cs"td||j|jk r.tdnd}d}d}d}i}x|D]\}}|dkr}|j|}nf|dkr|j|}nH|dkr|j|j}n$|dkr|j|j}n|j|||RRMtDEFAULT_ENCODINGRRRRRRRRRRRRRRRRRRRRRRRRRRR(((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRns8           #     cBsneZdZd edZdZdZdZdZ dZ dZ dZ d Z d ZRS( s4Good for XHTML, bad for tolerance of incorrect HTML.cCs'tjj|tj|||dS(N(t HTMLParserRR(RRfRS((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRscCs@ytjj||Wn"tjk r;}t|nXdS(N(RtfeedtHTMLParseErrorR(RRWtexc((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRscCstj||dS(N(RR(RR((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyt start_optionscCstj|dS(N(RR(R((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyt end_optionscCsnyt|d|}WnFtk r_yt|d|}Wntk rQqjX||n X||dS(Ntstart_tdo_(tgetattrRe(RttagRR((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pythandle_starttags  cCs6yt|d|}Wntk r*nX|dS(Ntend_(RRe(RRR((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyt handle_endtags  cCs |j|S(N(R(RR]((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRXscCs|S(N((RR]((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRscCs|S(N((RR((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pytunescape_attrs_if_requiredscCstjj||jdS(N(RtcloseR(R((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRsN(R=R>RRMRRRRRRRRXRRR(((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRs       t_AbstractSgmllibParsercBsDeZdZejdZdZdZdZdZ RS(cCstj||dS(N(RR(RR((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyt do_optionss3&(?:([a-zA-Z][-.a-zA-Z0-9]*)|#(x?[0-9a-fA-F]+))(;?)cCstd||j|jS(Ns&%s;(RXRR(RR]((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pytconvert_entityrefscCstd||jS(Ns%s(RKR(RR]((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pytconvert_charrefscCs|S(N((RR]((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRscCs|S(N((RR((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRs( R=R>RREtcompiletentity_or_charrefRRRR(((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRs     cBs/eZdZdedZdZdZRS(s4Good for tolerance of incorrect HTML, bad for XHTML.cCs'tjj|tj|||dS(N(t _sgmllib_copyt SGMLParserRR(RRfRS((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRscCs@ytjj||Wn"tjk r;}t|nXdS(N(RRRtSGMLParseErrorR(RRWR((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRscCstjj||jdS(N(RRRR(R((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRsN(R=R>RRMRRRR(((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRs t_AbstractBSFormParsercBs8eZdZdedZdZdZdZRS(cCs'tj||||jj|dS(N(RRt bs_base_class(RRfRS((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRscCs'tj|||jj||dS(N(RRR(RRW((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRscCs@y|jj||Wn"tjk r;}t|nXdS(N(RRRRR(RRWR((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyR scCs|jj||jdS(N(RRR(R((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRsN( R=R>RMRRRRRR(((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRs   tRobustFormParsercBseZdZejZRS(s.Tries to be highly tolerant of incorrect HTML.(R=R>Rt_beautifulsoupt BeautifulSoupR(((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRstNestingRobustFormParsercBseZdZejZRS(sTries to be highly tolerant of incorrect HTML. Different from RobustFormParser in that it more often guesses nesting above missing end tags (see BeautifulSoup docs). (R=R>RRtICantBelieveItsBeautifulSoupR(((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRsc Cs:t||j|t|||t|d|d|d| S(sSIdentical to ParseResponse, except that: 1. The returned list contains an extra item. The first form in the list contains all controls not contained in any FORM element. 2. The arguments ignore_errors and backwards_compat have been removed. 3. Backwards-compatibility mode (backwards_compat=True) is not available. t_urljoint _urlparset _urlunparse(t _ParseFileExtgeturlR,( tresponsetselect_defaulttform_parser_classt request_classRfRSR R R ((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyR-sc Cs4t|||t|||t|d|d|d| S(sOIdentical to ParseFile, except that: 1. The returned list contains an extra item. The first form in the list contains all controls not contained in any FORM element. 2. The arguments ignore_errors and backwards_compat have been removed. 3. Backwards-compatibility mode (backwards_compat=True) is not available. R R R (R R,( tfiletbase_uriRRRRfRSR R R ((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyROs cOst|}t||||S(N(R R(RCRR'R(tfh((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyt ParseStringqs cOst||j||dS(s~ Parse HTTP response and return a list of HTMLForm instances. The return value of mechanize.urlopen can be conveniently passed to this function as the response parameter. mechanize.ParseError is raised on parse errors. response: file-like object (supporting read() method) with a method geturl(), returning the URI of the HTTP response select_default: for multiple-selection SELECT controls and RADIO controls, pick the first item as the default if none are selected in the HTML form_parser_class: class to instantiate and use to pass request_class: class to return from .click() method (default is mechanize.Request) entitydefs: mapping like {"&": "&", ...} containing HTML entity definitions (a sensible default is used) encoding: character encoding used for encoding numeric character references when matching link text. mechanize does not attempt to find the encoding in a META HTTP-EQUIV attribute in the document itself (mechanize, for example, does do that and will pass the correct value to mechanize using this parameter). backwards_compat: boolean that determines whether the returned HTMLForm objects are backwards-compatible with old code. If backwards_compat is true: - ClientForm 0.1 code will continue to work as before. - Label searches that do not specify a nr (number or count) will always get the first match, even if other controls match. If backwards_compat is False, label searches that have ambiguous results will raise an AmbiguityError. - Item label matching is done by strict string comparison rather than substring matching. - De-selecting individual list items is allowed even if the Item is disabled. The backwards_compat argument will be removed in a future release. Pass a true value for select_default if you want the behaviour specified by RFC 1866 (the HTML 2.0 standard), which is to select the first item in a RADIO or multiple-selection SELECT control if none were selected in the HTML. Most browsers (including Microsoft Internet Explorer (IE) and Netscape Navigator) instead leave all items unselected in these cases. The W3C HTML 4.0 standard leaves this behaviour undefined in the case of multiple-selection SELECT controls, but insists that at least one RADIO button should be checked at all times, in contradiction to browser behaviour. There is a choice of parsers. mechanize.XHTMLCompatibleFormParser (uses HTMLParser.HTMLParser) works best for XHTML, mechanize.FormParser (uses bundled copy of sgmllib.SGMLParser) (the default) works better for ordinary grubby HTML. Note that HTMLParser is only available in Python 2.2 and later. You can pass your own class in here as a hack to work around bad HTML, but at your own risk: there is no well-defined interface. i(R R(RR'R(((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRus(R%RC(R((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRs(R=R>RR)R+R(((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRs   cCs-|jd}|dk r%t|SdSdS(NR(RLRMR(RRC((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyt _get_labels  cBs}eZdZd dZdZdZdZdZdZ dZ dZ d Z d Z d Zd ZRS(s An HTML form control. An HTMLForm contains a sequence of Controls. The Controls in an HTMLForm are accessed using the HTMLForm.find_control method or the HTMLForm.controls attribute. Control instances are usually constructed using the ParseFile / ParseResponse functions. If you use those functions, you can ignore the rest of this paragraph. A Control is only properly initialised after the fixup method has been called. In fact, this is only strictly necessary for ListControl instances. This is necessary because ListControls are built up from ListControls each containing only a single item, and their initial value(s) can only be known after the sequence is complete. The types and values that are acceptable for assignment to the value attribute are defined by subclasses. If the disabled attribute is true, this represents the state typically represented by browsers by 'greying out' a control. If the disabled attribute is true, the Control will raise AttributeError if an attempt is made to change its value. In addition, the control will not be considered 'successful' as defined by the W3C HTML 4 standard -- ie. it will contribute no data to the return value of the HTMLForm.click* methods. To enable a control, set the disabled attribute to a false value. If the readonly attribute is true, the Control will raise AttributeError if an attempt is made to change its value. To make a control writable, set the readonly attribute to a false value. All controls have the disabled and readonly attributes, not only those that may have the HTML attributes of the same names. On assignment to the value attribute, the following exceptions are raised: TypeError, AttributeError (if the value attribute should not be assigned to, because the control is disabled, for example) and ValueError. If the name or value attributes are None, or the value is an empty list, or if the control is disabled, the control is not successful. Public attributes: type: string describing type of control (see the keys of the HTMLForm.type2class dictionary for the allowable values) (readonly) name: name of control (readonly) value: current value of control (subclasses may allow a single value, a sequence of values, or either) disabled: disabled state readonly: readonly state id: value of id HTML attribute cCs tdS(s type: string describing type of control (see the keys of the HTMLForm.type2class dictionary for the allowable values) name: control name attrs: HTML attributes of control's HTML element N(tNotImplementedError(RRNR]RR((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRUscCs||_|jj|dS(N(t_formRR(RR#((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyt add_to_form_s cCsdS(N((R((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRcscCs tdS(N(R-(Rtkind((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyt is_of_kindfscCs tdS(N(R-(R((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pytcleariscCs tdS(N(R-(RR]((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyR)lscCs tdS(N(R-(RR]R((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyR+mscCs,g|jD]\}}}||f^q S(sMReturn list of (key, value) pairs suitable for passing to urlencode. (t_totally_ordered_pairs(RRxtktv((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pytpairsoscCs tdS(sReturn list of (key, value, index) tuples. Like pairs, but allows preserving correct ordering even where several controls are involved. N(R-(R((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyR3tscCsF|j}|jdd|d|jdd}|j|dS(s9Write data for a subitem of this control to a MimeWriter.sContent-Dispositionsform-data; name="%s"iRiN(RRRR(RtmwR]Rtmw2tf((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyt_write_mime_data}s   cCs tdS(N(R-(R((s4/scratch/rashmi/Condor_Script/src/mechanize/_form.pyRscCsTg}|jr"|j|jn|jrP|j|jjj|jdn|S(sReturn all labels (Label instances) for this control. If the control was surrounded by a