file io - Python iterate to remove newlines takes too long. Can my code be optimized better? -


i have few lines of python code reads file (1 column x 900 rows), creates json object list. problem running when trying remove newlines each item in list. here few examples:

this code takes approximately 11 seconds read (900 lines):

fh = open(wholefname) fc = fh.read() fh.close() 

this code takes approximately 17 minutes:

fh = open(wholefname) fc = fh.read() fh.close() markerarray = fc.splitlines() 

if, instead, use "fc = fh.readlines", each item in list has "\n" @ end. have tried placing .split() in json_list def at: "lst.append(pn.split())".

this full code looks like:

def json_list(list):   lst = []   pn in list:     lst.append(pn)   return json.dumps(lst)  fh = open(wholefname) fc = fh.read() fh.close() markerarray = fc.splitlines()  print json_list( markerarray ) 

i running jython python 2.6.2** on windows.

**edit: apologize, running python 2.5.2 on windows.

if remove function call (which useless), should faster.

with open(wholefname) fh:     json.dumps(fh.read().splitlines()) 

another thing aware of that, in python 2, there 2 versions of json module: 1 written in pure python, other in c. jython, know of, doesn't support c extensions, you're using python version, slower. best write code in java.


Comments