Recode bytes which cannot be decoded in utf-8 in python -

-

reading in txt files - there 1 byte causing me issues encode:

    open(input_filename_and_director, 'rb') f:         r = unicodecsv.reader(f, delimiter="|")

results in error message:

   unicodedecodeerror: 'utf8' codec can't decode byte 0xc3 in position 26: invalid continuation byte

is there anyway specify how want these bytes handled (i.e. read byte in character?)

depending upon want, try using unicodecsv.reader(f, delimiter="|", errors='replace') or unicodecsv.reader(f, delimiter="|", errors='ignore'). unicodecsv passes through errors parameter unicode encoding. see unicode or here more information.


Comments

Post a Comment

Popular posts from this blog

node.js - Mongoose: Cast to ObjectId failed for value on newly created object after setting the value -

-
this question has answer here: what's mongoose error cast objectid failed value xxx @ path “_id”? 7 answers even though has been asked many times because entity has no valid objectid field. i came across situation objectid set value can casted objectid still error. the schema: var serviceschema = parammongoose.schema({ servicename:{ type: parammongoose.schema.types.objectid , ref: 'servicenames',required:true } }); pseudo code: servicefromdb=new service({servicename:'some name'}); servicefromdb.servicename='000000000000000000000001'; servicefromdb.save(function(paramerror,paramdata){ if(paramerror){ console.log('but but...',servicefromdb,paramerror) } }); the output of code is: but but... { servicename: 000000000000000000000001, _id: 55079a90286f49280364f78b } { [casterror: cast objectid faile...
Read more

[C++][SFML 2.2] Strange Performance Issues - Moving Mouse Lowers CPU Usage -

-
i've been working on making 2d map editor past 2 weeks or so, , i've run strange problem. trying optimize code, example, making functions should've been functions begin with, did made cpu usage go through roof. i've tried commenting out different sections of code thought culprits (scrolling, rendering, other calculations) no avail. think issue might trying call things within different functions, sake of making modular possible, i'd functionality. the other major change i've made since file i'm comparing i've ported variables external cpp file, since having variables global makes lot easier use within different functions. simple issue, can't life of me understand why happens. with original code, around 2-3% cpu usage. new code, around 20-35%, but, if move mouse around, it'll drop around 6-8%. tl;dr: after trying optimize code using functions, i've somehow killed performance, , found strange quirk moving mouse reduces cpu usage.. /...
Read more

javascript - React + webpack: 'process.env' is undefined -

-
Image
i'm trying run hot dev server on our site webpack; site uses reactjs, has code in it: if (\"production\" !== process.env.node_env) // etc when not running hot-swap it's fine, hot-swap, gets run, resulting in error: typeerror: process.env undefined the code looks this: the project modelled after https://github.com/webpack/react-starter does work; question is; error have made in config file and/or how go looking error when 'production' compilation works fine? i've posted gist of webpack config file . in webpack config, there 2 options can affect process.env : when specify config.target (see config.target ) when define process.env variable via defineplugin looking @ code, looks process.env might undefined when both options.prerender , options.minimize false . you fix using environment defines process.env (ex: node ), or using defineplugin assign default value variable yourself.
Read more