python - Unzip my_file.zip pulled from s3 using boto -

-

i trying use boto open .zip file have in s3. trying work data directly, want avoid creating temporary files. 

in [201]: import stringio  in [202]: import boto  in [203]: conn = boto.connect_s3()  in [204]: my_bucket = conn.get_bucket('my_bucket')  in [205]: my_list = [ele ele in my_bucket.list('my_file.zip')]  in [206]: f = stringio.stringio()  in [207]: my_list[0].get_file(f)  in [208]: f.seek(0)

if file not zipped use:

my_content = my_list[0].get_contents_as_string()

but since zipped, getting garbage.

an answer question want (i borrowed bit of attempt it) using gzip, can't find using zip. tried using zipfilezipfile, read, extract , extractall methods don't seem want.

you should python module gzip :

https://docs.python.org/2/library/gzip.html

you should able stringio gzip. .

from boto.s3.connection import s3connection import gzip stringio import stringio  s3conn = s3connection() # assuming .boto has been setup bucket = s3conn.get_bucket('my_bucket') my_list = [gzip.gzipfile(fileobj=(stringio(ele.get_contents_as_string()))) ele in bucket.list()] #for readability pulled out item in my_list:     item.read()

for readability list comprehension should broken - followed original posting compare.

good luck!


Comments

Post a Comment

Popular posts from this blog

node.js - Mongoose: Cast to ObjectId failed for value on newly created object after setting the value -

-
this question has answer here: what's mongoose error cast objectid failed value xxx @ path “_id”? 7 answers even though has been asked many times because entity has no valid objectid field. i came across situation objectid set value can casted objectid still error. the schema: var serviceschema = parammongoose.schema({ servicename:{ type: parammongoose.schema.types.objectid , ref: 'servicenames',required:true } }); pseudo code: servicefromdb=new service({servicename:'some name'}); servicefromdb.servicename='000000000000000000000001'; servicefromdb.save(function(paramerror,paramdata){ if(paramerror){ console.log('but but...',servicefromdb,paramerror) } }); the output of code is: but but... { servicename: 000000000000000000000001, _id: 55079a90286f49280364f78b } { [casterror: cast objectid faile...
Read more

how does one get csharp-sqlite to throw exceptions for duplicates or foreign key constraint violations -

-
i making use of csharp-sqlite. in sqlite database have uniqueness , foreign key constraints. when write "duplicate" table exception not raised , way found check exception use following: var error = sqlitecommand.getlasterror(); i hoping able following: var sqlstring = "some insert sql"; var command = sqlitecommand(sqlstring); command.parameters.add(... parameter...); using (var sqlconnection = new sqliteconnection(connectionstring)) { try { command.executenonquery(); } catch (exception ex) { exception } { sqlconnection.close(); } } the "catch" block never hit, though var error = sqlitecommand.getlasterror(); does return description of error expecting. have idea how solve this, or standard practice using csharp-sqlite library?
Read more

Simple Angular 2 project fails 'Unexpected reserved word' -

-
i trying create simple angular2 project based on ng2do project. using quickstart , have following code... <script src="js/quickstart/dist/es6-shim.js"></script> .... import {component, template, bootstrap, foreach} 'js/quickstart/angular2/angular2'; this gives me following error... uncaught syntaxerror: unexpected reserved word what missing? don't know if helps have been having issues day realised because starting code script reference <script src="app.js"> </script> instead of using <script> system.import('app'); </script> hope helps using in tsconfig { "version": "1.5.0-beta", "compileroptions": { "target": "es6", "module": "amd", "declaration": false, "noimplicitany": false, "removecomments": true, "nolib": fal...
Read more