I exported the one of the csv file I wan to get some information using csvkit. but csvkit return the error like below:
'cp949' codec can't encode character u'\ufeff' in position 5: illegal multibyte sequence
BOM(https://en.wikipedia.org/wiki/Byte_order_mark) characters in the exported files. and I wrote the script to remove the BOM characters
import codecs
import sys
import os
try:
with open("%s.new" % sys.argv[1], 'w') as new:
with open(sys.argv[1], 'r') as f:
for line in f:
new.write(unicode(line.replace(codecs.BOM_UTF8, ''), 'utf-8'))
os.rename(sys.argv[1], "%s.bak" % sys.argv[1])
os.rename("%s.new" % sys.argv[1], sys.argv[1])
except Exception as e:
print(e)
Comments !