Posts tagged 'chunk'

Luca Lanziani

Read file in chunks in bash

How to read a file in chunks. You can optionally specify the starting point and the chunk size.

#!/bin/bash

function read_file_in_chunks { # FILENAME [START_FROM_LINE] [CHUNK_SIZE]
   FILENAME="$1" && shift
   START_FROM_LINE="$1" && shift || START_FROM_LINE=1
   CHUNK_SIZE="$1" && shift || CHUNK_SIZE=10

   FILE_LINES=$(wc -l < "${FILENAME}")
   FILE_LINE=${START_FROM_LINE}

   while [[ ${FILE_LINE} -le ${FILE_LINES} ]]; do
       CHUNK ...