symfony - Goutte Scrape Login to https Secure Website -

-

so i'm trying use goutte login https website following error:

curl error 60: ssl certificate problem: unable local issuer certificate 500 internal server error - requestexception 1 linked exception: ringexception

and code creator of goutte says use:

use goutte\client;  $client = new client();  $crawler = $client->request('get', 'http://github.com/'); $crawler = $client->click($crawler->selectlink('sign in')->link()); $form = $crawler->selectbutton('sign in')->form(); $crawler = $client->submit($form, array('login' => 'fabpot', 'password' =>     'xxxxxx')); $crawler->filter('.flash-error')->each(function ($node) {     print $node->text()."\n"; });

or here's come code symfony recommends: 

use goutte\client;  // make real request external site $client = new client(); $crawler = $client->request('get', 'https://github.com/login');  // select form , fill in values $form = $crawler->selectbutton('log in')->form(); $form['login'] = 'symfonyfan'; $form['password'] = 'anypass';  // submit form $crawler = $client->submit($form);

the thing neither of them work, error posted above. can, log in using code written in past question i've asked: curl scrape parse/find specific content

i want use symfony/goutte login in, scraping data need easier. or suggestions please? thanks!

adding following code fixes error (curl configuration):

    // make real request external site     $client = new client();     $client->getclient()->setdefaultoption('config/curl/'.curlopt_ssl_verifyhost, false);     $client->getclient()->setdefaultoption('config/curl/'.curlopt_ssl_verifypeer, false);     $crawler = $client->request('get', 'https://github.com/login');

but error occurs: 

the current node list empty. 500 internal server error - invalidargumentexception

once again, i'm using goutte symfony , default code test task, such logging https github.

the fix previous error node list empty github login page button says "sign in" , not submit or login on button. unfortunately, goutte api isn't clear if $form = $crawler->selectbutton('sign in')->form(); refers html name attribute or actual plain text of button. it's plain text; confusing. after more research of poorly documented api, ended following code works: 

// make real request external site $client = new client(); $client->getclient()->setdefaultoption('config/curl/'.curlopt_ssl_verifyhost, false); $client->getclient()->setdefaultoption('config/curl/'.curlopt_ssl_verifypeer, false); $crawler = $client->request('get', 'https://github.com/login');  // select form , fill in values $form = $crawler->selectbutton('sign in')->form(); $form['login'] = 'symfonyfan'; $form['password'] = 'anypass';  // submit form $crawler = $client->submit($form); echo $crawler->html();

Comments

Post a Comment

Popular posts from this blog

node.js - Mongoose: Cast to ObjectId failed for value on newly created object after setting the value -

-
this question has answer here: what's mongoose error cast objectid failed value xxx @ path “_id”? 7 answers even though has been asked many times because entity has no valid objectid field. i came across situation objectid set value can casted objectid still error. the schema: var serviceschema = parammongoose.schema({ servicename:{ type: parammongoose.schema.types.objectid , ref: 'servicenames',required:true } }); pseudo code: servicefromdb=new service({servicename:'some name'}); servicefromdb.servicename='000000000000000000000001'; servicefromdb.save(function(paramerror,paramdata){ if(paramerror){ console.log('but but...',servicefromdb,paramerror) } }); the output of code is: but but... { servicename: 000000000000000000000001, _id: 55079a90286f49280364f78b } { [casterror: cast objectid faile...
Read more

gradle error "Cannot convert the provided notation to a File or URI" -

-
i have defined 2 system properties in gradle.properties: systemprop.builddir=build systemprop.cfgdir=build\\cfg and have following task defined in build.gradle: task clean(group:'clean',description:'clean project') << { ant.sequential { delete(dir: system.properties.builddir) mkdir(dir: system.properties.builddir) delete(dir: system.properties.cfgdir) mkdir(dir: system.properties.cfgdir) } } this generates following error: execution failed task ':clean'. > cannot convert provided notation file or uri: {dir=build}. following types/formats supported: - string or charsequence path, e.g 'src/main/java' or '/usr/include' - string or charsequence uri, e.g 'file:/usr/include' - file instance. - uri or url instance. but equivalent block of code not generate error , works expected: task clean(group:'clean',description:'clean project') << { ...
Read more

python - NameError: name 'subprocess' is not defined -

-
i'm trying make rng website: http://jamesdotcom.com/?p=417 . when run following code: captureimage = subprocess.popen( [ "fswebcam", "-r", "356x292", "-d", "/dev/video0", "static.jpg", "--skip", "10" ], stdout=devnull, stderr=devnull) captureimage.communicate() i following error message: nameerror: name 'subprocess' not defined what should do? thanks have imported first? import subprocess captureimage = subprocess.popen(["fswebcam", ...) subprocess included in python standard library since python 2.4. should not necessary install it.
Read more