12.03.2007

why i hate computers...

... because they *should* help you automate things, but it never seems to work out that way.

I've been doing data analysis on large batches of data files (100's, not 10's, not 1000's) lately, and while I've written an IDL program to do the fitting, plotting, and output of parameters I need, there remained a few tasks that I couldn't figure out how to automate:


  1. prepare the batch file (unfortunately, the files are not entirely sequential, so it's not that easy to automate... I could pull some trick with getting a file list and going from there... but...)
  2. convert all of the .eps files that IDL outputs to .pdf so that I can actually use/view/print them


I decided not to bother about the batch file, it didn't really take that long, and once made I save it and never touch it again. BUT the only way I seemed to be able to convert the eps to pdf was to open all the eps files in Preview (it converts them on opening) and then save each of them. This is fine for a few eps (or ps) files... but when you have a 136 staring you in the face and know there are more to come... and that each one involves pressing command-w, enter, enter, you want to find a way that avoids causing tendonitis. Seriously.

First I turned to Automator (I'm a Mac user... unfortunately my solutions will be less than helpful for Linux users, and nearly useless for Windows users... sorry!). To be fair, I have actually managed to get Automator to do some useful things recently, but usually my encounters with the cute little robot leave me feeling rather let down. A quick perusal turned up what I thought was true anyhow, that Automator has no built in way to convert eps (or ps) to pdf. I also checked the Apple Automator downloads and Automator World to no avail.

So I thought, hey, I've got unix commands up the wazoo on here, there *must* be one that can convert eps to pdf...

pstopdf

excellent. So I pull up the man page, figure out how to use it, test it on a couple of files, and merrily enter:

pstopdf *.eps

uuh... geeee... that was fast. A quick "ls" showed me why... it only did the first file. Bugger. A bit of frigging around later, it was determined that pstopdf does not take multiple file arguments.

Ok. That stinks, but maybe now Automator can do something for me using the action "Run Shell Script". Yes, that's it, I'll grab the filenames with a "Find Finder Items" input it to "Run Shell Script" to run the pstopdf command, and voila, there will be 136 pdf's waiting for me. Right? Well not so simple, but yes, eventually. The two main quirks I had to overcome were:

  • Apparently at some point the file output and file input of "Find Finder Items" and "Run Shell Script" became mismatched. Oddly this can be solved by inserting a "Label Finder Items" between (you can select none, so it appears to do nothing) which mysteriously does indeed change the file listing in the correct manner.
  • When using the "Run Shell Script" action: do not enter anything into the script box until after you selected how you wish to pass the input from the previous action. That little drop down will insert the necessary script fragments you need to work with, but only if the box is empty. Since I tend to dive in, I of course immediately put my command in before selecting this, and got very frustrated very quickly.
  • Also, since the pstopdf command only takes one input at a time (and I couldn't get it to work with stdin, though the man page claims it should function) I used the "Pass input: as arguments" option and placed the pstopdf command inside the loop with "$f" as the argument where the input file should go. If you have a command that takes multiple inputs, you *should* be able to use "$@" outside the loop.



Long story short, I'll make available to you, the fruits of my frustration, all for the low, low price of *FREE* (err, uhh, and listening to my rant)... better yet, I'll include my other workflows that I've been using:


Since some people were having trouble downloading the workflows above, I've put the workflow to convert eps to pdf and to combine pdfs into a zip file for download: PDFworkflows.zip



Is that all? Oh no... no, no, my friends. I started many hours ago, this morning, with IDL opening into an xterm completely UNCONFIGURED for my needs. Let's look at my "needs":

  1. Something more readable for my not yet caffinated eyes than black on white (I like a nice soothing green on charcoal... reminds me of that old Apple//c we had when I was a kid)
  2. More pressing: a bigger window (ok, I can drag and resize, but sometimes funky things happen when I do that)
  3. And pure luxury would be a SCROLLBAR... yes I like to scroll, especially when I'm running batches of 136 files and I'd like to look at some of that output.


Is that too much to ask? I have my X11 configured to open an xterm that is more pleasing on the eyes, but since IDL launches it's own xterm from a mini-app, and I couldn't find where it actually did that anywhere I was stuck with what it gave me. Until today people, until today. All one needs to do is create a default xterm style you are happy with and then any program launching it's own xterm will use this default style. The trick is simple: create a .Xdefaults file in your home directory. The bits of code I used were:


XTerm*background:grey15
XTerm*foreground:PaleGreen
XTerm*vt100*geometry:160x48
XTerm*scrollBar:on
XTerm*scrollKey:on
XTerm*jumpScroll:on
XTerm*saveLines:500
XTerm*cursorColorBackground:DimGray
XTerm*cursorColor:DarkSeaGreen3
XTerm*highlight:LemonChiffon
XTerm*borderColor:SlateGrey


I got most of the commands from here and there is an xterm color table here. Now it looks like this:



Now go make a beautiful, scrollable xterm and make me proud... I have to believe I didn't waste my day in vain.

And convert some eps files to pdf while you're at it. Hell, convert a whole bunch of them and put them together into one gigantic pdf!

Because today, I got my computer to do what I wanted it to do... It just took a really long time.

9 comments:

Anonymous said...

Thank you! Now I am really happy :)
An automatic way to convert eps to pdf is something I have been looking for for over a year.

Wooden Gate said...

I felt really proud at the fact that I came really close to the same conclusion you found with automator. Now I feel really dumb because when I click on the link you provided to the automator action you posted, I can not find the file that I need to download. There is one that downloads as document.wflow.txt, but can not be opened in automator. I as well need to automate the process of converting an Art Explosion library of eps files to pdf so they will work with quicklook.

woodengate@mac.com

physics*chick said...

oh dang. sorry about that wooden gate. I think it's that the unix system i've got the files on makes the workflow show up as a folder (I presume it is trying to outsmart us and show us the package contents?). for starters try just right (option) clicking on the link and download contents which should give you the workflow package.

If that doesn't work, I'll go back and make zip or .dmg files out of them for download.

let me know!

Dirk said...

I was having the same problem too and also thought Automator! Then I also found out that the `Change type of image' does not accept eps files, and it ignores them silently! Very frustrating.

However, if you are going to use a commandline program, then you could as well just use the unix shell completely as
for f in *.eps; do echo $f; epstopdf $f; done
(Or whatever command you were wanting to use instead of epstopdf to do the conversion; and the echo $f is just to have some progress indication on your terminal.)

Anonymous said...

If you need to run a program for a number of arguments, use 'xargs':

pstopdf *.eps

becomes

ls *.eps | xargs pstopdf

or more reliably (allows for large numbers of files as well as "unprintable" characters in filenames):

find . -maxdepth 1 -iname '*.eps' -print0 | xargs -0 pstopdf


x-man-page://xargs will tell you more...

David said...

Thank you so much for doing the grunt work on this, physics*chick!

I had this same problem, wrestled with automator, even tried the "record" feature of automator to get it to record my action of opening the eps, letting it convert, then saving it; but that only resulted in automator hanging when I gave it just 20 or so files. Then I googled around and found your blog. Your automator workflow works like a charm. The nice thing is, once the eps files are pdf's, they can be converted to any other file type via the "convert image type" automator action.

You've definitely inspired me to get more unix savvy. Thanks again!

Tommy said...

Thanks Mad Scientist! I've been looking for the better part of a year to do just this!

Anonymous said...

thanks works liked a charm, after i updated the workflow elements using Snow Leopard.

Awesome!!!

Anonymous said...

Thanks a lot, for this Automator Workflow.

I've been searching for a while for such a useful script.
Because I have to convert a lot eps to pdf.

Before I knew this script I opened every single eps with preview and saved it as a pdf manually.
Now this goes much faster.