Month: January 2015

Writing lists to a file

After creating my list from the previous post I needed to save this to a text file to send back to the person who had requested the information. I just wrote a standard loop over the list but this got me thinking about a better way to do this as it’s a common task. As it turns out not really but it did bring up a few titbits worth mentioning.

If you are from a PERL background you might be tempted to use a single write and a join to turn the list into a string as follows

file.write("\n".join(mylist))

Don’t. This creates a new string in memory from the list and then writes that. Ok if it is a small list but a huge waste of memory for larger ones.

For the more experienced Python user, using a generator produces arguably a cleaner syntax and saves a line of typing. As with all generators, you can make it conditonal by adding an if clause to the end.

with open(r'x:\path\to\file.txt','w') as txtfile:
  txtfile.writelines( "%s\n" % item for item in mylist )

That covers off the text file, but there may be a better option depending upon what is going to be done with the resulting file. I’ll finish the following couple of examples.

# if the file is going to Excel, maybe a CSV file
import csv
with open(r'x:\path\to\file.csv','w') as csvfile:
  csv = csv.writer(csvfile)
  for item in mylist:
    csv.writerow([item])

# or how about json if it is going to another program
import json # Python 2.5 use simplejson
with open(r'x:\path\to\file.json','w') as jsonfile:
  json.dump(mylist,jsonfile)

See the Python docs for the csv module and json module.

Processing file names

I seem to have spent a lot of time working with files this month. My task today was summarise the output from an inventory tool. This tool (well vb script) had created a lot of text files with the computer name and a date serial in the name. What I wanted is this information in a CSV file to compare to our asset list.

There are lots of ways of doing this but as the computer name is variable length but otherwise the file is known I used a regular expression. Regular expressions can get complicated but it this case I’m just looking for any letter, number, underscore or dash followed by .example.com (for my computer name) then an underscore followed by a 12 digit datetime serial. The rest of the name is irrelevant.

I am also using grouping by enclosing the computer name and datetime serial in parentheses. This allows me to return the matched details using group().

For the code below to work you would need to define a function formatdate to turn the datetime serial into something more readable. I’ve left this out to improve the clarity of the example.

import os, re, csv
prog = re.compile(r"([a-zA-Z0-9_\-]+).example.com_([0-9]{12})")

def getinfo ( instr ):
    res = prog.match(instr)
    if res:
        return (res.group(1),formatdate(res.group(2))]

with open('names.csv','w', newline='') as csvfile:
    csvwriter = csv.writer(csvfile)
    csvwriter.writerow(['Computer','Date']) # header
    for file in os.listdir(r'\\server\path\to\inventories'):
        fileinfo = getinfo(file)
        if fileinfo:
            csvwriter.writerow(fileinfo)

The natural progression would be to list comprehension to create a list and then write this all out in one go using writerows instead. However getting this to handle cases where the match failed resulted in code ugly code. If I work a way around this I’ll include it.

If you needed to include sub-directories as well then you could use os.walk instead of os.listdir then loop of the files list returned.

Empty directories

Today I got asked if it was possible to recurse through a folder structure and find all the empty directories within that structure. It sounded like a simple job for os.walk, and it was.

The os.walk recurses a directory structure returning a 3-tuple for each directory (including the starting directory) containing the directory path, a list of all the directories from that path and a list of all the files from that path. An empty directory has no folders and no files in it so this gives us an easy condition

import os
for dirpath,dirs,files in os.walk(r'\\server\path\to\root\directory'):
 if (len(dirs)+len(files)) == 0:
   print(dirpath)

That lists all the empty directories to the screen but it would be more useful to have them in a list. Here the power of list comprehension comes into its own allowing us to do this with just one line of code

import os
emptydirs = [dirpath for dirpath,dirs,files in os.walk(r'\\server\path\to\root\directory') if (len(dirs)+len(files)) == 0]