Reading Files in Bash

    I came across a small but interesting difference in ways of reading files in bash, and specifically processing a file line by line.

    I had a small function to read property files and splitting each line in a key and a value and suddenly a bug cropped up. This worked fine when reading existing property files. But when introducing a space right of the =, it failed. It made two “lines” out of “foo=bar zoot”, while one line out of “foo=bar”.

    for line in $(< ${file}); do
      echo ${line}
    done

    It’s our old friend IFS (internal field separator) at work here, with its default value of “" in bash. So we unset it and bob's our uncle.

    prev_ifs=${IFS}
    IFS=''
    
    for line in $(< ${file}); do
      echo ${line}
    done
    
    IFS=${prev_ifs}

    (It’s nice to preserve the value of IFS and reset it after you’re done. If you’r in a function you can of course local it instead)

    This code works fine without any IFS twiddling, though:

    while read -r line; do
      echo ${line}
    done < "$file"

    There are manny more ways to read files in bash of course. The lesson is of course that I should have tested this better when I first wrote it :-)

    Other recent posts

  • 2016-06-12 » The Kubernetes Wars: Day 7
  • 2016-06-04 » The Kubernetes Wars: Day 3
  • 2016-06-02 » The Kubernetes Wars: Day 0
  • 2016-03-11 » Talk Friday: Microservice Architecture
  • 2015-09-10 » Deploy Overlay in Metric Graphs
  • 2015-01-29 » Cache invalidation: explained
  • 2014-10-15 » A Usable and Robust Private Npm Repo
  • 2013-10-20 » The Metrics Approach
  • 2012-11-15 » Rapid Java Deployment

Meta

Knut Haugen [Knu:t Hæugen] is a Norwegian software developer with a soft spot for dynamic languages, DevOps and anything to with developer testing. Writes mostly java by day (and some by night too, actually) and ruby and javascript by night. I am an agile methodology geek with bias on Lean and Kanban. I race mountain bikes and drink proper beer.

You can find me on twitter or some kind of electric letter by the address of knuthaug ætt gmail