ó BÜXQc@s/dZddlZddlZddlZddlZddlZddlmZddlZddl Z ddl Z ddl Z ddl Z ddl mZdefd„ƒYZdefd„ƒYZd efd „ƒYZd „Zd dd „ƒYZde jfd„ƒYZdefd„ƒYZdS(sUStateful programmatic WWW navigation, after Perl's WWW::Mechanize. Copyright 2003-2006 John J. Lee Copyright 2003 Andy Lester (original Perl code) This code is free software; you can redistribute it and/or modify it under the terms of the BSD or ZPL 2.1 licenses (see the file COPYING.txt included with the distribution). iÿÿÿÿN(tDefaultFactory(t UserAgentBasetBrowserStateErrorcBseZRS((t__name__t __module__(((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyRstLinkNotFoundErrorcBseZRS((RR(((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyRstFormNotFoundErrorcBseZRS((RR(((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyRscCs>tj|ƒ}tjdkr:|jdƒr:|d}n|S(Ntnts///i(turllibt pathname2urltostnamet startswith(tpathturlpath((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pytsanepathname2urls tHistorycBs;eZdZd„Zd„Zd„Zd„Zd„ZRS(sT Though this will become public, the implied interface is not yet stable. cCs g|_dS(N(t_history(tself((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyt__init__)scCs|jj||fƒdS(N(Rtappend(Rtrequesttresponse((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pytadd+scCsr|}x_|dks!|dkrgy|jjƒ\}}Wntk rYtdƒ‚nX|d8}q W||fS(Nisalready at start of historyi(tNoneRtpopt IndexErrorR(Rtnt _responseRR((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pytback-s cCs |j2dS(N(R(R((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pytclear6scCs>x0|jD]%\}}|dk r |jƒq q W|j2dS(N(RRtclose(RRR((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyR8s (RRt__doc__RRRRR(((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyR#s    tHTTPRefererProcessorcBseZd„ZeZRS(cCs.t|dƒr*|jj|dtƒ}n|S(Nt redirect_dicttorigin_request(thasattrtparentt_add_referer_headertFalse(RR((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyt http_request@s (RRR(t https_request(((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyR!?s tBrowserc BsÏeZdZejejƒZeed|S| r\|jjdƒ r\|S|jrÒ|dkrÒ|dko†|dk rÒtj|jjƒƒ}|d d}tj|ƒ}|j d|ƒn|S(NthttpthttpstRefereriÿÿÿÿ(shttpshttps(shttpshttps(N( RRtget_typet has_headerR-t_rfc3986turlsplitt get_full_urlt urlunsplittadd_unredirected_header(RRR#tschemetoriginal_schemetpartstreferer((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyR&¥s     cCs|j||dtd|ƒS(s7Open a URL without visiting it. Browser state (including request, response, history, forms and links) is left unchanged by calling this function. The interface is the same as for .open(). This is useful for things like fetching images. See also .retrieve(). tvisitttimeout(t _mech_openR'(RturltdataRZ((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyt open_novisit¹scCs|j||d|ƒS(NRZ(R[(RR\R]RZ((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pytopenÉsc Cs„y |jWnrtk rtj|ƒd \}}|dkr€|jdkr^tdƒ‚ntj|jjƒ|ƒ}q€nX|j ||||ƒ}|j }|dkr¶t }n|rÏ|j ||ƒnt } yt j|||ƒ} Wn7tjk r'} t} | jdkr‚n| } nX|rS|j| tƒtj|jƒ} n| dk rqtj| ƒ} n| s€| ‚n| S(Nis8can't fetch relative reference: not viewing any document(RRtAttributeErrorRPRQRRRturljoinR8R.RYR,t_visit_requestRR_turllib2t HTTPErrorR'tfpR3tcopytupgrade_response( RR\R]tupdate_historyRYRZRUt authorityRtsuccessRterror((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyR[Ís>    "      cCsg}|jd|jjƒ|jrC|jd|jjƒƒn |jdƒ|jrv|jdt|jƒƒn|jdƒdj|ƒS(Ns<%s s visiting %ss(not visiting a URL)s selected form: %s t>t(Rt __class__RRR8R6tstrtjoin(Rttext((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyt__str__s    cCstj|jƒS(s¨Return a copy of the current response. The returned object has the same interface as the object returned by .open() (or mechanize.urlopen()). (RfR(R((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyRscCs/ttjj|ƒƒ}d|}|j|ƒS(Nsfile://(RR R tabspathR_(RtfilenameR R\((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pytopen_local_files cCs|j|tƒdS(s”Replace current response with (a copy of) response. response may be None. This is intended mostly for HTML-preprocessing. N(R3R,(RR((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyR7scCs±|dkp6t|dƒo6t|dƒo6t|dƒsHtdƒ‚nd|_|dk rotj|ƒ}n|r”|jdk r”|jjƒn||_|jj|ƒdS(NtinfoR8treadsnot a response object( RR$t ValueErrorR6RRgRR1R7(RRt close_current((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyR3%s    cCsH|dkr$tj|jƒƒ}n|j|tƒ|j|tƒdS(s£Visit the response, as if it had been .open()ed. Unlike .set_response(), this updates history rather than replacing the current response. N(RR.R/R8RbR,R3R'(RRR((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pytvisit_response6s cCsf|jdk r|jjƒn|jdk rP|rP|jj|j|jƒnd|_||_dS(N(RRRRRR(RRRh((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyRbAs  cCs+|jdkrtdƒ‚n|jjƒS(sGet URL of current document.snot viewing any documentN(RRRR8(R((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyR8KscCsS|jdkrtdƒ‚n|jdk r=|jjƒn|j|jdtƒS(s4Reload current document, and return response object.sno URL has yet been .open()edRhN(RRRRRR[R'(R((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyR9Qs icCsm|jdk r|jjƒn|jj||jƒ\|_}|j|ƒ|js`|jƒSt j |ƒS(s{Go back n steps in history, and return response object. n: go back this number of steps (default 1 step) N( RRRRRRR7t read_completeR9Rf(RRR((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyRYs!   cCs|jjƒdS(N(RR(R((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyR:gscCs‹|jdkrtdƒ‚n|jjƒdkrBtdƒ‚n|jdj}|jƒ}|jƒ}||d<|j ||jƒdS( s´Request to set a cookie. Note that it is NOT necessary to call this method under ordinary circumstances: cookie handling is normally entirely automatic. The intended use case is rather to simulate the setting of a cookie by client script in a web page (e.g. JavaScript). In that case, use of this method is necessary because mechanize currently does not support JavaScript, VBScript, etc. The cookie is added in the same way as if it had arrived with the current response, as a result of the current request. This means that, for example, if it is not appropriate to set the cookie based on the current request, no cookie will be set. The cookie will be returned automatically with subsequent responses made by the Browser instance whenever that's appropriate. cookie_string should be a valid value of the Set-Cookie header. For example: browser.set_cookie( "sid=abcdef; expires=Wednesday, 09-Nov-06 23:12:40 GMT") Currently, this method does not allow for adding RFC 2986 cookies. This limitation will be lifted if anybody requests it. snot viewing any documentRKRLs0can't set cookie for non-HTTP/HTTPS transactionst_cookiess Set-cookieN(shttpshttps( RRRRRNt _ua_handlerst cookiejarRRvtextract_cookies(Rt cookie_stringR~Rtheaders((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyR;js   cKsH|jƒstdƒ‚n|jjƒ}|r@|j||S|SdS(s4Return iterable over links (mechanize.Link objects).snot viewing HTMLN(R>RR1R<t _filter_links(RtkwdsR<((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyR<’s  cCs(|jƒstdƒ‚n|jjƒS(stReturn iterable over forms. The returned form objects implement the mechanize.HTMLForm interface. snot viewing HTML(R>RR1R=(R((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyR=œs cCs%|jƒstdƒ‚n|jjS(sØReturn the global form object, or None if the factory implementation did not supply one. The "global" form object contains all controls that are not descendants of any FORM element. The returned form object implements the mechanize.HTMLForm interface. This is a separate method since the global form is not regarded as part of the sequence of forms in the document -- mostly for backwards-compatibility. snot viewing HTML(R>RR1t global_form(R((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyR„¦s cCs(|jdkrtdƒ‚n|jjS(s7Return whether the current response contains HTML data.snot viewing any documentN(RRRR1tis_html(R((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyR>¸scCs(|jdkrtdƒ‚n|jjS(Nsnot viewing any document(RRRR1R?(R((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyR?¾scCs%|jƒstdƒ‚n|jjS(sÂReturn title, or None if there is no title element in the document. Treatment of any tag children of attempts to follow Firefox and IE (currently, tags are preserved). snot viewing HTML(R>RR1R@(R((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyR@Ãs c Cs|jƒstdƒ‚n|dkrN|dkrN|dkrNtdƒ‚n|jj}|dkr—|dkr—|dk r—||ƒr—||_dS|}xù|jƒD]f}|dk rÑ||jkrÑqªn|dk rð||ƒ rðqªn|r|d8}qªn||_PqªWg}|dk r:|j d|ƒn|dk rZ|j d|ƒn|dk rz|j d|ƒndj |ƒ}t d |ƒ‚dS( sySelect an HTML form for input. This is a bit like giving a form the "input focus" in a browser. If a form is selected, the Browser object supports the HTMLForm interface, so you can call methods like .set_value(), .set(), and .click(). Another way to select a form is to assign to the .form attribute. The form assigned should be one of the objects returned by the .forms() method. At least one of the name, predicate and nr arguments must be supplied. If no matching form is found, mechanize.FormNotFoundError is raised. If name is specified, then the form must have the indicated name. If predicate is specified, then the form must match that function. The predicate function is passed the HTMLForm as its single argument, and should return a boolean value indicating whether the form matched. nr, if supplied, is the sequence number of the form (where 0 is the first). Note that control 0 is the first form matching all the other arguments (if supplied); it is not necessarily the first control in the form. The "global form" (consisting of all form controls not contained in any FORM element) is considered not to be part of this sequence and to have no name, so will not be matched unless both name and nr are None. snot viewing HTMLs6at least one argument must be supplied to specify formNis name '%s's predicate %ssnr %ds, sno form matching ( R>RRRxR1R„R6R=R RRpR(RR t predicatetnrR„torig_nrR6t description((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyRAÎs< $        cOs=|jƒstdƒ‚n|jj||Ž}|j|ƒS(s/See mechanize.HTMLForm.click for documentation.snot viewing HTML(R>RR6RBR&(RtargsRƒR((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyRBs cOs|j|j||ŽƒS(sSubmit current form. Arguments are as for mechanize.HTMLForm.click(). Return value is same as for Browser.open(). (R_RB(RRŠRƒ((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyRCscKsg|jƒstdƒ‚n|s3|j|}n|rHtdƒ‚n|j|jƒ}|j|ƒS(s§Find a link and return a Request object for it. Arguments are as for .find_link(), except that a link may be supplied as the first argument. snot viewing HTMLs2either pass a Link, or keyword arguments, not both(R>RRFRxR2t absolute_urlR&(RtlinkRƒR((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyRDs  cKs|j|j||ƒS(s‡Find a link and .open() it. Arguments are as for .click_link(). Return value is same as for Browser.open(). (R_RD(RRŒRƒ((s9/scratch/rashmi/Condor_Script/src/mechanize/_mechanize.pyRE1scKsDy#|j|jjƒ|jƒSWntk r?tƒ‚nXdS(s«Find a link in current page. Links are returned as mechanize.Link objects. # Return third link that .search()-matches the regexp "python" # (by ".search()-matches", I mean that the regular expression method # .search() is used, rather than .match()). find_link(text_regex=re.compile("python"), nr=2) # Return first http link in the current page that points to somewhere # on python.org whose link text (after tags have been removed) is # exactly "monty python". find_link(text="monty python", url_regex=re.compile("http.*python.org")) # Return first link with exactly three HTML attributes. find_link(predicate=lambda link: len(link.attrs) == 3) Links include anchors (), image maps (), and frames (,