Data Wrangling for 3-D Geologic Modeling of Oil Reservoirs


I'm a geologist at Amoco Exploration and Production in Houston, using Sun and SGI workstations for petroleum well log analysis and 3-D geologic modeling of oil reservoirs. Additionally I have a Compaq Pentium II PC running NT for company email and MS Office stuff that also functions as my intranet webserver to make my UNIX stuff available real-time on the company intranet.

I work with lots of data in many different formats and I am always needing to reformat / merge / extract big data files to facilitate transfer between different platforms and software packages. My coworkers and I historically used various text editors in UNIX and Windows (including using MS Excel) to manuually reformat, edit and merge data sets (shudder).

A few years back I discovered awk, sed, csh and more recently sh scripts. These really sped up my work process, but I could never get them to work as well together as I would have liked. Back in June I was talking to the help desk of one of our software vendors about using shell scripting to extract and reformat data from the output reports of the 3-D geologic software package. I was concerned about the calculational precision of sh scripts. The support guy told me he used Perl for these types of data extraction and reformating tasks. I'd heard of Perl; I knew it was the basis of cgi-bin scripts on the web, but that was the extent of my knowledge.

So that night at home, on my four year old 68K Mac (Performa 630CD), I did an Internet search on Perl and found MacPerl: Power and Ease. I called around and actually found one copy at a local computer superstore. The guy at the store seemed amazed that I wanted it - said he didn't get much call for Mac programming books. I picked up the book - took it home - loaded it up and started teaching myself Perl over the weekend.

The next week, I overheard my coworkers discussing the best way to get data from one data source, merge it with data from a second data source (with wellname cross-referenced to ID number) and reformat for loading into yet a third software package. As usual, Excel lookup tables were discussed as were various text editors. I thought "This is a job for MacPerl". So I emailed test copies of the various data formats to myself at home and began work on my first Perl script. I stayed up late several nights playing with MacPerl and finally got the script to work the way I wanted it to. I emailed it back to work and (once I got the correct path to Perl5 on our UNIX system) it ran - and it ran fast!

As luck would have it, my company's technical training group was offering its second-ever in-house Perl class the following week. I signed up and during the class I continued to work on my script (rlogged into my normal workstation), emailing it home to continue development with MacPerl, and emailing it back to work. This first script was definitely "write only", but it worked and saved several hours of work.

Since July, I have written dozens of Perl scripts, saving myself and my coworkers many hours (and days) of time and effort. I have built an in-house website to share my results with others throughout the company and provide links to numerous other Perl sites. I even have Perl scripts that create HTML pages cataloging and documenting the scripts I've written.

I've since gotten the "Llama" and "Camel" books to go with my "Dragonlord". My scripts are now much more readable and better documented. They still seem pretty basic compared to other Perl scripts I've read, but they get the job done and they really are incredibly fast. My coworkers are amazed at how much time these scripts save and I have finally achieved fame and respect (well ... so I exaggerated about the "respect" :-). The best part is having the ability to develop and test scripts at home with MacPerl (on an old 68K Mac at that) and then having those scripts perform flawlessly in the UNIX environment at work.

The content, organization, and instructional style of MacPerl: Power and Ease is excellent, as is the wealth of info on the included MacPerl CD-ROM. MacPerl itself is an exceptional value in terms of performance, scalability, and compatibility with UNIX and Win32 versions of Perl5. The ability to use BBEdit from within MacPerl and the excellent online documentation also accessible from within MacPerl are real time savers.

MacPerl: Power and Ease is one of the best books I've purchased for my Mac.

Chris Bergman
Geologist / Petrophysicist
Amoco Exploration and Production
Houston, Texas