Application development

Recipe Crates Categories
Parse command line arguments clap-badge cat-command-line-badge
Decompress a tarball flate2-badge tar-badge cat-compression-badge
Compress a directory into a tarball flate2-badge tar-badge cat-compression-badge
Decompress a tarball while removing a prefix from the paths flate2-badge tar-badge cat-compression-badge
Avoid writing and reading from a same file same_file-badge cat-filesystem-badge
Find loops for a given path same_file-badge cat-filesystem-badge
Recursively find duplicate file names walkdir-badge cat-filesystem-badge
Recursively find all files with given predicate walkdir-badge cat-filesystem-badge
Traverse directories while skipping dotfiles walkdir-badge cat-filesystem-badge
Recursively calculate file sizes at given depth walkdir-badge cat-filesystem-badge
Find all png files recursively glob-badge cat-filesystem-badge
Find all files with given pattern ignoring filename case glob-badge cat-filesystem-badge

Parse command line arguments

clap-badge cat-command-line-badge

extern crate clap;

use clap::{Arg, App};

fn main() {
    // Define command line arguments.
    let matches = App::new("My Test Program")
        .version("0.1.0")
        .author("Hackerman Jones <hckrmnjones@hack.gov>")
        .about("Teaches argument parsing")
        .arg(Arg::with_name("file")
                 .short("f")
                 .long("file")
                 .takes_value(true)
                 .help("A cool file"))
        .arg(Arg::with_name("num")
                 .short("n")
                 .long("number")
                 .takes_value(true)
                 .help("Five less than your favorite number"))
        .get_matches();

    // Get value for file, or default to 'input.txt'.
    let myfile = matches.value_of("file").unwrap_or("input.txt");
    println!("The file passed is: {}", myfile);

    // Get value for num if present, and try parsing it as i32.
    let num_str = matches.value_of("num");
    match num_str {
        None => println!("No idea what your favorite number is."),
        Some(s) => {
            match s.parse::<i32>() {
                Ok(n) => println!("Your favorite number must be {}.", n + 5),
                Err(_) => println!("That's not a number! {}", s),
            }
        }
    }
}

The clap crate is a simple-to-use, efficient, and full-featured library for parsing command line arguments and subcommands when writing console/terminal applications.

The application can describe the structure of its command-line interface using clap's builder style. The documentation gives two other possible ways to instantiate an application.

In the builder style, with_name is the unique identifier that value_of will use to retrieve the value passed. The short and long options control the flag the user will be expected to type; short flags look like -f and long flags look like --file.

Usage information is generated by clap. The usage for the example application looks like this.

My Test Program 0.1.0
Hackerman Jones <hckrmnjones@hack.gov>
Teaches argument parsing

USAGE:
    testing [OPTIONS]

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

OPTIONS:
    -f, --file <file>     A cool file
    -n, --number <num>    Five less than your favorite number

We can test the application by running a command like the following.

$ cargo run -- -f myfile.txt -n 251

The output is:

The file passed is: myfile.txt
Your favorite number must be 256.

Decompress a tarball

flate2-badge tar-badge cat-compression-badge

Decompress (GzDecoder) and extract (Archive::unpack) all files from a compressed tarball named archive.tar.gz located in the current working directory.

# #[macro_use]
# extern crate error_chain;
extern crate flate2;
extern crate tar;

use std::fs::File;
use flate2::read::GzDecoder;
use tar::Archive;
#
# error_chain! {
#     foreign_links {
#         Io(std::io::Error);
#     }
# }

fn run() -> Result<()> {
    let path = "archive.tar.gz";

    // Open a compressed tarball
    let tar_gz = File::open(path)?;
    // Decompress it
    let tar = GzDecoder::new(tar_gz)?;
    // Load the archive from the tarball
    let mut archive = Archive::new(tar);
    // Unpack the archive inside curent working directory
    archive.unpack(".")?;

    Ok(())
}
#
# quick_main!(run);

Compress a directory into tarball

flate2-badge tar-badge cat-compression-badge

Compresses /var/log directory into archive.tar.gz.

Creates a File wrapped in GzEncoder and tar::Builder.
Adds contents of /var/log directory recursively into the archive under backup/logspath with Builder::append_dir_all. GzEncoder is responsible for transparently compressing the data prior to writing it into archive.tar.gz.

# #[macro_use]
# extern crate error_chain;
extern crate tar;
extern crate flate2;
#
# error_chain! {
#     foreign_links {
#         Io(std::io::Error);
#     }
# }

use std::fs::File;
use flate2::Compression;
use flate2::write::GzEncoder;

fn run() -> Result<()> {
    let tar_gz = File::create("archive.tar.gz")?;
    let enc = GzEncoder::new(tar_gz, Compression::Default);
    let mut tar = tar::Builder::new(enc);
    tar.append_dir_all("backup/logs", "/var/log")?;
    Ok(())
}
#
# quick_main!(run);

Decompress a tarball while removing a prefix from the paths

flate2-badge tar-badge cat-compression-badge

Strip a path prefix from the entries of a tarball before unpacking them.

We iterate over the Archive::entries, using Path::strip_prefix to remove the specified path prefix (bundle/logs) before extracting the tar::Entry via Entry::unpack.

# #[macro_use]
# extern crate error_chain;
extern crate flate2;
extern crate tar;
#
# error_chain! {
#     foreign_links {
#         Io(std::io::Error);
#         StripPrefixError(::std::path::StripPrefixError);
#     }
# }

use std::fs::File;
use std::path::PathBuf;
use flate2::read::GzDecoder;
use tar::Archive;

fn run() -> Result<()> {
    let file = File::open("archive.tar.gz")?;
    let mut archive = Archive::new(GzDecoder::new(file)?);
    let prefix = "bundle/logs";
    let entries = archive
        .entries()?
        .filter_map(|e| e.ok())
        .map(|mut entry| -> Result<PathBuf> {
            // Need to get owned data to break the borrow loop
            let path = entry.path()?.strip_prefix(prefix)?.to_owned();
            entry.unpack(&path)?;
            Ok(path)
        });

    println!("Extracted the following files:");
    for path in entries.filter_map(|e| e.ok()) {
        println!("> {}", path.display());
    }
    Ok(())
}
#
# quick_main!(run);

Avoid writing and reading from a same file

same_file-badge cat-filesystem-badge

Use same_file::Handle to a file that can be tested for equality with other handles. In this example, the handles of file to be read from and to be written to are tested for equality.

cargo run

will display the contents of the file if the two files are not same and

cargo run >> ./new.txt

will display the error (because the two files are same).

# #[macro_use]
# extern crate error_chain;
extern crate same_file;

use same_file::Handle;
use std::path::Path;
use std::fs::File;
use std::io::{BufRead, BufReader};
#
# error_chain! {
#     foreign_links {
#          IOError(::std::io::Error);
#     }
# }

fn run() -> Result<()> {
    let path_to_read = Path::new("new.txt");

    let stdout_handle = Handle::stdout()?;
    let handle = Handle::from_path(path_to_read)?;

    if stdout_handle == handle {
        bail!("You are reading and writing to the same file");
    } else {
        let file = File::open(&path_to_read)?;
        let file = BufReader::new(file);
        for (num, line) in file.lines().enumerate() {
            println!("{} : {}", num, line?.to_uppercase());
        }
    }

    Ok(())
}
#
# quick_main!(run);

Find loops for a given path

same_file-badge cat-filesystem-badge

Use same_file::is_same_file to detect loops for a given path. For example, a loop could be created on a Unix system via symlinks:

mkdir -p /tmp/foo/bar/baz
ln -s /tmp/foo/  /tmp/foo/bar/baz/qux

Then, running the following would assert that there exists a loop.

extern crate same_file;

use std::io;
use std::path::{Path, PathBuf};
use same_file::is_same_file;

// Check this path against all of its parents
fn contains_loop<P: AsRef<Path>>(path: P) -> io::Result<Option<(PathBuf, PathBuf)>> {
    let path = path.as_ref();
    let mut path_buf = path.to_path_buf();
    while path_buf.pop() {
        if is_same_file(&path_buf, path)? {
            return Ok(Some((path_buf, path.to_path_buf())));
        } else if let Some(looped_paths) = contains_loop(&path_buf)? {
            return Ok(Some(looped_paths));
        }
    }
    return Ok(None);
}

fn main() {
    assert_eq!(
        contains_loop("/tmp/foo/bar/baz/qux/bar/baz").unwrap(),
        Some((
            PathBuf::from("/tmp/foo"),
            PathBuf::from("/tmp/foo/bar/baz/qux")
        ))
    );
}

Recursively find duplicate file names

walkdir-badge cat-filesystem-badge

Find recursively in the current directory duplicate filenames, printing them only once.

extern crate walkdir;

use std::collections::HashMap;
use walkdir::WalkDir;

fn main() {
    // Counters indexed by filenames
    let mut filenames = HashMap::new();

    // List recusively all files in the current directory filtering out
    // directories and files not accessible (permission denied)
    for entry in WalkDir::new(".")
            .into_iter()
            .filter_map(Result::ok)
            .filter(|e| !e.file_type().is_dir()) {
        // Get entry's filename
        let f_name = String::from(entry.file_name().to_string_lossy());
        // Get or initialize the counter
        let counter = filenames.entry(f_name.clone()).or_insert(0);
        // Update the counter
        *counter += 1;

        if *counter == 2 {
            println!("{}", f_name);
        }
    }
}

Recursively find all files with given predicate

walkdir-badge cat-filesystem-badge

Find JSON files modified within the last day in the current directory. Using follow_links ensures symbolic links are followed like they were normal directories and files.

# #[macro_use]
# extern crate error_chain;
extern crate walkdir;

use walkdir::WalkDir;
#
# error_chain! {
#     foreign_links {
#         WalkDir(walkdir::Error);
#         Io(std::io::Error);
#         SystemTime(std::time::SystemTimeError);
#     }
# }

fn run() -> Result<()> {
    // List recusively all accessible files in the current directory
    for entry in WalkDir::new(".")
            .follow_links(true)
            .into_iter()
            .filter_map(|e| e.ok()) {
        // Get entry's filename
        let f_name = entry.file_name().to_string_lossy();
        // Get entry's modified time
        let sec = entry.metadata()?.modified()?;

        // Print JSON files modified within the last day
        if f_name.ends_with(".json") && sec.elapsed()?.as_secs() < 86400 {
            println!("{}", f_name);
        }
    }

    Ok(())
}
#
# quick_main!(run);

Traverse directories while skipping dotfiles

walkdir-badge cat-filesystem-badge

Uses WalkDirIterator::filter_entry to descend recursively into entries passing the is_not_hidden predicate thus skipping hidden files and directories whereas Iterator::filter would be applied to each WalkDir::DirEntry even if the parent is a hidden directory.

Root dir "." is yielded due to WalkDir::depth usage in is_not_hidden predicate.

extern crate walkdir;

use walkdir::{DirEntry, WalkDir, WalkDirIterator};

fn is_not_hidden(entry: &DirEntry) -> bool {
    entry
         .file_name()
         .to_str()
         .map(|s| entry.depth() == 0 || !s.starts_with("."))
         .unwrap_or(false)
}

fn main() {
    for result in WalkDir::new(".")
            .into_iter()
            .filter_entry(|e| is_not_hidden(e))
            .filter_map(|v| v.ok()) {
        println!("{}", result.path().display());
    }
}

Recursively calculate file sizes at given depth

walkdir-badge cat-filesystem-badge

Recursion depth can be flexibly set by WalkDir::min_depth & WalkDir::max_depth methods. In this example we sum all file sizes to 3 subfolders depth, ignoring files in the root folder at the same time.

extern crate walkdir;

use walkdir::WalkDir;

fn main() {
    let total_size = WalkDir::new(".")
        .min_depth(1)
        .max_depth(3)
        .into_iter()
        .filter_map(|entry| entry.ok()) // Files, we have access to
        .filter_map(|entry| entry.metadata().ok()) // Get metadata
        .filter(|metadata| metadata.is_file()) // Filter out directories
        .fold(0, |acc, m| acc + m.len()); // Accumulate sizes

    println!("Total size: {} bytes.", total_size);
}

Find all png files recursively

glob-badge cat-filesystem-badge

Recursively find all PNG files in the current directory. In this case, the ** pattern matches the current directory and all subdirectories.

You can also use the ** pattern for any directory, not just the current one. For example, /media/**/*.png would match all PNGs in media and it's subdirectories.

# #[macro_use]
# extern crate error_chain;
extern crate glob;

use glob::glob;
#
# error_chain! {
#     foreign_links {
#         Glob(glob::GlobError);
#         Pattern(glob::PatternError);
#     }
# }

fn run() -> Result<()> {
    for entry in glob("**/*.png")? {
        println!("{}", entry?.display());
    }

    Ok(())
}
#
# quick_main!(run);

Find all files with given pattern ignoring filename case.

glob-badge cat-filesystem-badge

Find all image files in the /media/ directory matching the img_[0-9]*.png pattern.

A custom MatchOptions struct is passed to the glob_with function making the glob pattern case insensitive while keeping the other options Default.

# #[macro_use]
# extern crate error_chain;
extern crate glob;

use glob::{glob_with, MatchOptions};
#
# error_chain! {
#     foreign_links {
#         Glob(glob::GlobError);
#         Pattern(glob::PatternError);
#     }
# }

fn run() -> Result<()> {
    let options = MatchOptions {
        case_sensitive: false,
        ..Default::default()
    };

    for entry in glob_with("/media/img_[0-9]*.png", &options)? {
        println!("{}", entry?.display());
    }

    Ok(())
}
#
# quick_main!(run);