AverageAzn247
Grandmaster Cheater
 Reputation: 34
Joined: 01 Oct 2007
Posts: 909
Location: Austin,TX with 72 virgins
|
 Posted: Wed Aug 15, 2012 12:35 pm Post subject: removing duplicates from a list |
 |
|
I am writing a program to remove duplicates from a list however i also want to remove duplicates that vary with extra white spaces like foo foo and foo foo. What I tried to doing was first removing all the duplicates using python's build in set. Then I tried creating two lists, one without any whitespaces and the original. Next I compared the two without any whitespaces and deleted duplicates. For some reason some duplicates still remain. why?
| Code: | import string
def main():
inputfile=raw_input("file name?: ")
file1=open(inputfile, "r")
output=raw_input("output: ")
sorted_list1=sort1(file1)
sorted_list2=sort2(sorted_list1)
write_file(sorted_list2,output)
def sort1(file1):
temp=""
list1=[]
for rawline in file1:#scan line by line
temp+=rawline#stores as string untill added
key=rawline.strip()#removes whitespace and tabs for checkign
if key == "endmodule":# if end of mod add to list
list1.append(temp)
#print temp
temp=""# clears the string until next mod
#print len(list1),"list 1"
list1=list(set(list1))
return list1
def sort2(sorted1):
temp=""
list1=sorted1
list1.sort()
list2=[]
for mod in list1:
list2.append(mod.strip())
list2.sort()
last=list2[-1]
for i in range(len(list2)-2,-1,-1):
if last == list2[i]:
#print list2[i]
del list2[i]
del list1[i]
else:
last = list2[i]
return list1
def write_file(sorted_list2,output):
file4=open(output,"w")
print "new module",len(sorted_list2)
for mod in sorted_list2:
file4.write(mod)
#print mod
file4.close()
main() |
_________________
| Waxxup wrote: | What are Night Elves?
A girl group? |
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
