Cookin' with Rust

This Rust Cookbook is a collection of simple examples that demonstrate good practices to accomplish common programming tasks, using the crates of the Rust ecosystem.

Read more about Rust Cookbook, including tips for how to read the book, how to use the examples, and notes on conventions.

Contributing

This project is intended to be easy for new Rust programmers to contribute to, and an easy to way get involved with the Rust community. It needs and welcomes help. For details see CONTRIBUTING.md.

Basics

Recipe Crates Categories
Read lines of strings from a file std-badge cat-filesystem-badge
Read and write integers in little-endian byte order byteorder-badge cat-encoding-badge
Generate random numbers rand-badge cat-science-badge
Generate random numbers within a range rand-badge cat-science-badge
Generate random numbers with given distribution rand-badge cat-science-badge
Generate random values of a custom type rand-badge cat-science-badge
Create random passwords from a set of alphanumeric characters rand-badge cat-os-badge
Create random passwords from a set of user-defined characters rand-badge cat-os-badge
Run an external command and process stdout regex-badge cat-os-badge cat-text-processing-badge
Run an external command passing it stdin and check for an error code std-badge cat-os-badge
Run piped external commands std-badge cat-os-badge
Redirect both stdout and stderr of child process to the same file std-badge cat-os-badge
Continuously process child process' outputs std-badge cat-os-badgecat-text-processing-badge
Filter a log file by matching multiple regular expressions regex-badge cat-text-processing-badge
Declare lazily evaluated constant lazy_static-badge cat-caching-badge cat-rust-patterns-badge
Maintain global mutable state lazy_static-badge cat-rust-patterns-badge
Verify and extract login from an email address regex-badge lazy_static-badge cat-text-processing-badge
Extract a list of unique #Hashtags from a text regex-badge lazy_static-badge cat-text-processing-badge
Replace all occurrences of one text pattern with another pattern. regex-badge lazy_static-badge cat-text-processing-badge
Extract phone numbers from text regex-badge cat-text-processing-badge
Calculate the SHA-256 digest of a file ring-badge data-encoding-badge cat-cryptography-badge
Sign and verify a message with an HMAC digest ring-badge cat-cryptography-badge
Salt and hash a password with PBKDF2 ring-badge data-encoding-badge cat-cryptography-badge
Define and operate on a type represented as a bitfield bitflags-badge cat-no-std-badge
Access a file randomly using a memory map memmap-badge cat-filesystem-badge
Check number of logical cpu cores num_cpus-badge cat-hardware-support-badge
Handle errors correctly in main error-chain-badge cat-rust-patterns-badge
Avoid discarding errors during error conversions error-chain-badge cat-rust-patterns-badge
Obtain backtrace of complex error scenarios error-chain-badge cat-rust-patterns-badge
Measure elapsed time std-badge cat-time-badge
Convert date to UNIX timestamp and vice versa chrono-badge cat-date-and-time-badge
Convert a local time to an another UTC timezone and vice versa chrono-badge cat-date-and-time-badge
Display formatted date and time chrono-badge cat-date-and-time-badge
Parse string into DateTime struct chrono-badge cat-date-and-time-badge
Perform checked date and time calculations chrono-badge cat-date-and-time-badge
Examine the date and time chrono-badge cat-date-and-time-badge
File names that have been modified in the last 24 hours for the working directory std-badge cat-filesystem-badge cat-os-badge

Encoding

Recipe Crates Categories
Serialize and deserialize unstructured JSON serde-json-badge cat-encoding-badge
Deserialize a TOML configuration file toml-badge cat-encoding-badge
Percent-encode a string url-badge cat-encoding-badge
Encode a string as application/x-www-form-urlencoded url-badge cat-encoding-badge
Encode and decode hex data-encoding-badge cat-encoding-badge
Encode and decode base64 base64-badge cat-encoding-badge
Read CSV records csv-badge cat-encoding-badge
Read CSV records with different delimiter csv-badge cat-encoding-badge
Filter CSV records matching a predicate csv-badge cat-encoding-badge
Handle invalid CSV data with Serde csv-badge serde-badge cat-encoding-badge
Serialize records to CSV csv-badge cat-encoding-badge
Serialize records to CSV using Serde csv-badge serde-badge cat-encoding-badge
Transform one column of a CSV file csv-badge serde-badge cat-encoding-badge
Get MIME type from string mime-badge cat-encoding-badge

Concurrency

Recipe Crates Categories
Mutate the elements of an array in parallel rayon-badge cat-concurrency-badge
Test in parallel if any or all elements of a collection match a given predicate rayon-badge cat-concurrency-badge
Search items using given predicate in parallel rayon-badge cat-concurrency-badge
Sort a vector in parallel rayon-badge rand-badge cat-concurrency-badge
Map-reduce in parallel rayon-badge cat-concurrency-badge
Generate jpg thumbnails in parallel rayon-badge glob-badge image-badge cat-concurrency-badgecat-filesystem-badge
Spawn a short-lived thread crossbeam-badge cat-concurrency-badge
Draw fractal dispatching work to a thread pool threadpool-badge num-badge num_cpus-badge image-badge cat-concurrency-badgecat-science-badgecat-rendering-badge
Calculate SHA1 sum of *.iso files concurrently threadpool-badge walkdir-badge num_cpus-badge ring-badge cat-concurrency-badgecat-filesystem-badge

Networking

Recipe Crates Categories
Parse a URL from a string to a Url type url-badge cat-net-badge
Create a base URL by removing path segments url-badge cat-net-badge
Create new URLs from a base URL url-badge cat-net-badge
Extract the URL origin (scheme / host / port) url-badge cat-net-badge
Remove fragment identifiers and query pairs from a URL url-badge cat-net-badge
Make a HTTP GET request reqwest-badge cat-net-badge
Download a file to a temporary directory reqwest-badge tempdir-badge cat-net-badge cat-filesystem-badge
Query the GitHub API reqwest-badge serde-badge cat-net-badge cat-encoding-badge
Consume a paginated RESTful API reqwest-badge serde-badge cat-net-badge cat-encoding-badge
Check if an API resource exists reqwest-badge cat-net-badge
Set custom headers and URL parameters for a REST request reqwest-badge hyper-badge url-badge cat-net-badge
Create and delete Gist with GitHub API reqwest-badge serde-badge cat-net-badge cat-encoding-badge
POST a file to paste-rs reqwest-badge cat-net-badge
Listen on unused port TCP/IP std-badge cat-net-badge
Extract all links from a webpage HTML reqwest-badge select-badge cat-net-badge
Check webpage for broken links reqwest-badge select-badge url-badge cat-net-badge
Extract all unique links from a MediaWiki markup reqwest-badge regex-badge cat-net-badge
Make a partial download with HTTP range headers reqwest-badge cat-net-badge
Handle a rate-limited API reqwest-badge hyper-badge cat-net-badge
Parse the MIME type of a HTTP response mime-badge reqwest-badge cat-net-badge cat-encoding-badge

Application development

Recipe Crates Categories
Parse command line arguments clap-badge cat-command-line-badge
Decompress a tarball flate2-badge tar-badge cat-compression-badge
Compress a directory into a tarball flate2-badge tar-badge cat-compression-badge
Decompress a tarball while removing a prefix from the paths flate2-badge tar-badge cat-compression-badge
Avoid writing and reading from a same file same_file-badge cat-filesystem-badge
Find loops for a given path same_file-badge cat-filesystem-badge
Recursively find duplicate file names walkdir-badge cat-filesystem-badge
Recursively find all files with given predicate walkdir-badge cat-filesystem-badge
Traverse directories while skipping dotfiles walkdir-badge cat-filesystem-badge
Recursively calculate file sizes at given depth walkdir-badge cat-filesystem-badge
Find all png files recursively glob-badge cat-filesystem-badge
Find all files with given pattern ignoring filename case glob-badge cat-filesystem-badge
Parse and increment a version string semver-badge cat-config-badge
Parse a complex version string semver-badge cat-config-badge
Check if given version is pre-release semver-badge cat-config-badge
Find the latest version satisfying given range semver-badge cat-config-badge
Check external command version for compatibility semver-badge cat-text-processing-badge cat-os-badge

Logging

Recipe Crates Categories
Log a debug message to the console log-badge env_logger-badge cat-debugging-badge
Log an error message to the console log-badge env_logger-badge cat-debugging-badge
Enable log levels per module log-badge env_logger-badge cat-debugging-badge
Log to stdout instead of stderr log-badge env_logger-badge cat-debugging-badge
Log messages with a custom logger log-badge cat-debugging-badge
Use a custom environment variable to set up logging log-badge env_logger-badge cat-debugging-badge
Include timestamp in log messages log-badge env_logger-badge chrono-badge cat-debugging-badge
Log to the Unix syslog log-badge syslog-badge cat-debugging-badge
Log messages to a custom location log-badge log4rs-badge cat-debugging-badge

Build Time Tooling

Recipe Crates Categories
Compile and link statically to a bundled C library cc-badge cat-development-tools-badge
Compile and link statically to a bundled C++ library cc-badge cat-development-tools-badge
Compile a C library while setting custom defines cc-badge cat-development-tools-badge

About "Cookin' with Rust"

Table of contents

Who this book is for

This cookbook is intended for new Rust programmers, so that they may quickly get an overview of the capabilities of the Rust crate ecosystem. It is also intended for experienced Rust programmers, who should find in the recipes an easy reminder of how to accomplish common tasks.

How to read this book

The cookbook index contains the full list of recipes, organized into a number of sections: "basics", "encoding", "concurrency", etc. The sections themselves are more or less ordered in progression, with later sections being more advanced, and occasionally building on concepts from earlier sections.

Within the index, each section contains a list of recipes. The recipes are simple statements of a task to accomplish, like "generate random numbers in a range"; and each recipe is tagged with badges indicating which crates they use, like rand-badge, and which categories on crates.io those crates belong to, like cat-science-badge.

New Rust programmers should be comfortable reading from the first section to the last, and doing so should give one a strong overview of the crate ecosystem. Click on the section header in the index, or in the sidebar to navigate to the page for that section of the book.

If you are simply looking for the solution to a simple task, the cookbook is today more difficult to navigate. The easiest way to find a specific recipe is to scan the index looking for the crates and categories one is interested in. From there, click on the name of the recipe to view it. This will improve in the future.

How to use the recipes

Recipes are designed to give you instant access to working code, along with a full explanation of what it is doing, and to guide you to further information.

All recipes in the cookbook are full, self contained programs, so that they may be copied directly into your own projects for experimentation. To do so follow the instructions below.

Consider this example for "generate random numbers within a range":

rand-badge cat-science-badge

extern crate rand;
use rand::Rng;

fn main() {
    let mut rng = rand::thread_rng();
    println!("Random f64: {}", rng.gen::<f64>());
}

To work with it locally we can run the following commands to create a new cargo project, and change to that directory:

cargo new my-example --bin
cd my-example

Now, we also need to add the necessary crates to Cargo.toml, as indicated by the crate badges, in this case just "rand". To do so, we'll use the cargo add command, which is provided by the cargo-edit crate, which we need to install first:

cargo install cargo-edit
cargo add rand

Now you can replace src/main.rs with the full contents of the example and run it:

cargo run

The crate badges that accompany the examples link to the crates' full documentation on docs.rs, and is often the next documentation you should read after deciding which crate suites your purpose.

A note about error handling

Error handling in Rust is robust when done correctly, but in today's Rust it requires a fair bit of boilerplate. Because of this one often sees Rust examples filled with unwrap calls instead of proper error handling.

Since these recipes are intended to be reused as-is and encourage best practices, they set up error handling correctly when there are Result types involved.

The basic pattern we use is to have a fn run() -> Result that acts like the "real" main function. We use the error-chain crate to make ? work within run.

The structure generally looks like:

#[macro_use]
extern crate error_chain;

use std::net::IpAddr;
use std::str;

error_chain! {
    foreign_links {
        Utf8(std::str::Utf8Error);
        AddrParse(std::net::AddrParseError);
    }
}

fn run() -> Result<()> {
    let bytes = b"2001:db8::1";

    // Bytes to string.
    let s = str::from_utf8(bytes)?;

    // String to IP address.
    let addr: IpAddr = s.parse()?;

    println!("{:?}", addr);
    Ok(())
}

quick_main!(run);

This is using the error_chain! macro to define a custom Error and Result type, along with automatic conversions from two standard library error types. The automatic conversions make the ? operator work. The quick_main! macro generates the actual main function and prints out the error if one occurred.

For the sake of readability error handling boilerplate is hidden by default like below. In order to read full contents click on the "expand" () button located in the top right corner of the snippet.

# #[macro_use]
# extern crate error_chain;
extern crate url;

use url::{Url, Position};
#
# error_chain! {
#     foreign_links {
#         UrlParse(url::ParseError);
#     }
# }

fn run() -> Result<()> {
    let parsed = Url::parse("https://httpbin.org/cookies/set?k2=v2&k1=v1")?;
    let cleaned: &str = &parsed[..Position::AfterPath];
    println!("cleaned: {}", cleaned);
    Ok(())
}
#
# quick_main!(run);

For more background on error handling in Rust, read this page of the Rust book and this blog post.

A note about crate representation

This cookbook is intended eventually to provide expansive coverage of the Rust crate ecosystem, but today is limited in scope while we get it bootstrapped and work on the presentation. Hopefully, starting from a small scope and slowly expanding will help the cookbook become a high-quality resource sooner, and allow it to maintain consistent quality levels as it grows.

At present the cookbook is focused on the standard library, and on "core", or "foundational", crates—those crates that make up the most common programming tasks, and that the rest of the ecosystem builds off of.

The cookbook is closely tied to the Rust Libz Blitz, a project to identify, and improve the quality of such crates, and so it largely defers crate selection to that project. Any crates that have already been evaluated as part of that process are in scope for the cookbook, as are crates that are pending evaluation.

Basics

Recipe Crates Categories
Read lines of strings from a file std-badge cat-filesystem-badge
Read and write integers in little-endian byte order byteorder-badge cat-encoding-badge
Generate random numbers rand-badge cat-science-badge
Generate random numbers within a range rand-badge cat-science-badge
Generate random numbers with given distribution rand-badge cat-science-badge
Generate random values of a custom type rand-badge cat-science-badge
Create random passwords from a set of alphanumeric characters rand-badge cat-os-badge
Create random passwords from a set of user-defined characters rand-badge cat-os-badge
Run an external command and process stdout regex-badge cat-os-badge cat-text-processing-badge
Run an external command passing it stdin and check for an error code regex-badge cat-os-badge cat-text-processing-badge
Run piped external commands std-badge cat-os-badge
Redirect both stdout and stderr of child process to the same file std-badge cat-os-badge
Continuously process child process' outputs std-badge cat-os-badgecat-text-processing-badge
Filter a log file by matching multiple regular expressions regex-badge cat-text-processing-badge
Declare lazily evaluated constant lazy_static-badge cat-caching-badge cat-rust-patterns-badge
Maintain global mutable state lazy_static-badge cat-rust-patterns-badge
Verify and extract login from an email address regex-badge lazy_static-badge cat-text-processing-badge
Extract a list of unique #Hashtags from a text regex-badge lazy_static-badge cat-text-processing-badge
Replace all occurrences of one text pattern with another pattern. regex-badge lazy_static-badge cat-text-processing-badge
Extract phone numbers from text regex-badge cat-text-processing-badge
Calculate the SHA-256 digest of a file ring-badge data-encoding-badge cat-cryptography-badge
Sign and verify a message with an HMAC digest ring-badge cat-cryptography-badge
Salt and hash a password with PBKDF2 ring-badge data-encoding-badge cat-cryptography-badge
Define and operate on a type represented as a bitfield bitflags-badge cat-no-std-badge
Access a file randomly using a memory map memmap-badge cat-filesystem-badge
Check number of logical cpu cores num_cpus-badge cat-hardware-support-badge
Handle errors correctly in main error-chain-badge cat-rust-patterns-badge
Avoid discarding errors during error conversions error-chain-badge cat-rust-patterns-badge
Obtain backtrace of complex error scenarios error-chain-badge cat-rust-patterns-badge
Measure elapsed time std-badge cat-time-badge
Convert date to UNIX timestamp and vice versa chrono-badge cat-date-and-time-badge
Convert a local time to an another UTC timezone and vice versa chrono-badge cat-date-and-time-badge
Display formatted date and time chrono-badge cat-date-and-time-badge
Parse string into DateTime struct chrono-badge cat-date-and-time-badge
Perform checked date and time calculations chrono-badge cat-date-and-time-badge
Examine the date and time chrono-badge cat-date-and-time-badge
File names that have been modified in the last 24 hours for the working directory std-badge cat-filesystem-badge cat-os-badge

Read lines of strings from a file

std-badge cat-filesystem-badge

Writes a three-line message to a file, then reads it back a line at a time with the Lines iterator created by BufRead::lines. BufRead is a trait, and the most common way to get one is from a BufReader, which is constructed from some type that implements Read, here a File. The File is opened for writing with File::create, and reading with File::open.

# #[macro_use]
# extern crate error_chain;
#
use std::fs::File;
use std::io::{Write, BufReader, BufRead};
#
# error_chain! {
#     foreign_links {
#         Io(std::io::Error);
#     }
# }

fn run() -> Result<()> {
    let path = "lines.txt";

    let mut output = File::create(path)?;
    write!(output, "Rust\n💖\nFun")?;

    let input = File::open(path)?;
    let buffered = BufReader::new(input);

    for line in buffered.lines() {
        println!("{}", line?);
    }

    Ok(())
}
#
# quick_main!(run);

Read and write integers in little-endian byte order

byteorder-badge cat-encoding-badge

# #[macro_use]
# extern crate error_chain;
extern crate byteorder;

use byteorder::{LittleEndian, ReadBytesExt, WriteBytesExt};

#[derive(Default, PartialEq, Debug)]
struct Payload {
    kind: u8,
    value: u16,
}
#
# error_chain! {
#     foreign_links {
#         Io(std::io::Error);
#     }
# }

fn run() -> Result<()> {
    let original_payload = Payload::default();
    let encoded_bytes = encode(&original_payload)?;
    let decoded_payload = decode(&encoded_bytes)?;
    assert_eq!(original_payload, decoded_payload);
    Ok(())
}

fn encode(payload: &Payload) -> Result<Vec<u8>> {
    let mut bytes = vec![];
    bytes.write_u8(payload.kind)?;
    bytes.write_u16::<LittleEndian>(payload.value)?;
    Ok(bytes)
}

fn decode(mut bytes: &[u8]) -> Result<Payload> {
    let payload = Payload {
        kind: bytes.read_u8()?,
        value: bytes.read_u16::<LittleEndian>()?,
    };
    Ok(payload)
}
#
# quick_main!(run);

Generate random numbers

rand-badge cat-science-badge

Generates random numbers with help of random-number generator rand::Rng obtained via rand::thread_rng.

extern crate rand;

use rand::Rng;

fn main() {
    // Each thread has an automatically-initialised random number generator:
    let mut rng = rand::thread_rng();

    // Integers are uniformly distributed over the type's whole range:
    let n1: u8 = rng.gen();
    let n2: u16 = rng.gen();
    println!("Random u8: {}", n1);
    println!("Random u16: {}", n2);
    println!("Random u32: {}", rng.gen::<u32>());
    println!("Random i32: {}", rng.gen::<i32>());

    // Floating point numbers are uniformly distributed in the half-open range [0, 1)
    println!("Random float: {}", rng.gen::<f64>());
}

Generate random numbers within a range

rand-badge cat-science-badge

Generates a random value within half-open [0, 10) range (not including 10) with Rng::gen_range.

extern crate rand;

use rand::Rng;

fn main() {
    let mut rng = rand::thread_rng();
    println!("Integer: {}", rng.gen_range(0, 10));
    println!("Float: {}", rng.gen_range(0.0, 10.0));
}

Alternatively, one can use Range to obtain values with uniform distribution. This has the same effect, but may be faster when repeatedly generating numbers in the same range.

extern crate rand;

use rand::distributions::{Range, Distribution};

fn main() {
    let mut rng = rand::thread_rng();
    let die = Range::new(1, 7);

    loop {
        let throw = die.sample(&mut rng);
        println!("Roll the die: {}", throw);
        if throw == 6 {
            break;
        }
    }
}

Generate random numbers with given distribution

rand-badge cat-science-badge

By default, random numbers are generated with uniform distribution. To generate numbers with other distributions you instantiate a distribution, then sample from that distribution using Distribution::sample with help of a random-number generator rand::Rng.

The distributions available are documented here. An example using the Normal distribution is shown below.

extern crate rand;

use rand::distributions::{Normal, Distribution};

fn main() {
    let mut rng = rand::thread_rng();

    // mean 2, standard deviation 3:
    let normal = Normal::new(2.0, 3.0);
    let v = normal.sample(&mut rng);
    println!("{} is from a N(2, 9) distribution", v)
}

Generate random values of a custom type

rand-badge cat-science-badge

Randomly generates a tuple (i32, bool, f64) and variable of user defined type Point. Implements the Distribution trait on type Point for Standard in order to allow random generation.

extern crate rand;

use rand::Rng;
use rand::distributions::{Distribution, Standard};

#[derive(Debug)]
struct Point {
    x: i32,
    y: i32,
}

impl Distribution<Point> for Standard {
    fn sample<R: Rng + ?Sized>(&self, rng: &mut R) -> Point {
        let (rand_x, rand_y) = rng.gen();
        Point {
            x: rand_x,
            y: rand_y,
        }
    }
}

fn main() {
    let mut rng = rand::thread_rng();
    let rand_tuple = rng.gen::<(i32, bool, f64)>();
    let rand_point: Point = rng.gen();
    println!("Random tuple: {:?}", rand_tuple);
    println!("Random Point: {:?}", rand_point);
}

Create random passwords from a set of alphanumeric characters

rand-badge cat-os-badge

Randomly generates a string of given length ASCII characters in the range A-Z, a-z, 0-9, with Alphanumeric sample.

extern crate rand;

use rand::{thread_rng, Rng};
use rand::distributions::Alphanumeric;

fn main() {
    let rand_string: String = thread_rng()
        .sample_iter(&Alphanumeric)
        .take(30)
        .collect();

    println!("{}", rand_string);
}

Create random passwords from a set of user-defined characters

rand-badge cat-os-badge

Randomly generates a string of given length ASCII characters with custom user-defined bytestring, with choose.

extern crate rand;

use rand::{thread_rng, Rng};

fn main() {
    const CHARSET: &[u8] =  b"ABCDEFGHIJKLMNOPQRSTUVWXYZ\
    abcdefghijklmnopqrstuvwxyz\
    0123456789)(*&^%$#@!~";

    let mut rng = thread_rng();
    let password: Option<String> = (0..30)
        .map(|_| Some(*rng.choose(CHARSET)? as char))
        .collect();

    println!("{:?}", password);
}

Run an external command and process stdout

regex-badge cat-os-badge cat-text-processing-badge

Runs git log --oneline as an external Command and inspects its Output using Regex to get the hash and message of the last 5 commits.

# #[macro_use]
# extern crate error_chain;
extern crate regex;

use std::process::Command;
use regex::Regex;
#
# error_chain!{
#     foreign_links {
#         Io(std::io::Error);
#         Regex(regex::Error);
#         Utf8(std::string::FromUtf8Error);
#     }
# }

#[derive(PartialEq, Default, Clone, Debug)]
struct Commit {
    hash: String,
    message: String,
}

fn run() -> Result<()> {
    let output = Command::new("git").arg("log").arg("--oneline").output()?;

    if !output.status.success() {
        bail!("Command executed with failing error code");
    }

    let pattern = Regex::new(r"(?x)
                               ([0-9a-fA-F]+) # commit hash
                               (.*)           # The commit message")?;

    String::from_utf8(output.stdout)?
        .lines()
        .filter_map(|line| pattern.captures(line))
        .map(|cap| {
                 Commit {
                     hash: cap[1].to_string(),
                     message: cap[2].trim().to_string(),
                 }
             })
        .take(5)
        .for_each(|x| println!("{:?}", x));

    Ok(())
}
#
# quick_main!(run);

Run an external command passing it stdin and check for an error code

std-badge cat-os-badge

Opens the python interpreter using an external Command and passes it a python statement for execution. Output of said statement is then parsed.

# #[macro_use]
# extern crate error_chain;
#
use std::collections::HashSet;
use std::io::Write;
use std::process::{Command, Stdio};
#
# error_chain!{
#     errors { CmdError }
#     foreign_links {
#         Io(std::io::Error);
#         Utf8(std::string::FromUtf8Error);
#     }
# }

fn run() -> Result<()> {
    let mut child = Command::new("python").stdin(Stdio::piped())
        .stderr(Stdio::piped())
        .stdout(Stdio::piped())
        .spawn()?;

    child.stdin
        .as_mut()
        .ok_or("Child process stdin has not been captured!")?
        .write_all(b"import this; copyright(); credits(); exit()")?;

    let output = child.wait_with_output()?;

    if output.status.success() {
        let raw_output = String::from_utf8(output.stdout)?;
        let words = raw_output.split_whitespace()
            .map(|s| s.to_lowercase())
            .collect::<HashSet<_>>();
        println!("Found {} unique words:", words.len());
        println!("{:#?}", words);
        Ok(())
    } else {
        let err = String::from_utf8(output.stderr)?;
        bail!("External command failed:\n {}", err)
    }
}
#
# quick_main!(run);

Run piped external commands

std-badge cat-os-badge

Shows up to the 10th biggest files and subdirectories in the current working directory. It is equivalent to run: du -ah . | sort -hr | head -n 10.

It spawns Unix processes which are represented as Commands. In order to capture the output of a child process it is necessary to create a new Stdio::piped between parent and child.

# #[macro_use]
# extern crate error_chain;
#
use std::process::{Command, Stdio};
#
# error_chain! {
#     foreign_links {
#         Io(std::io::Error);
#         Utf8(std::string::FromUtf8Error);
#     }
# }

fn run() -> Result<()> {
    let directory = std::env::current_dir()?;
    let du_output = Command::new("du")
        .arg("-ah")
        .arg(&directory)
        .stdout(Stdio::piped())
        .spawn()?
        .stdout
        .ok_or_else(|| "Could not capture `du` standard output.")?;

    let sort_output = Command::new("sort")
        .arg("-hr")
        .stdin(du_output)
        .stdout(Stdio::piped())
        .spawn()?
        .stdout
        .ok_or_else(|| "Could not capture `sort` standard output.")?;

    let head_output = Command::new("head")
        .args(&["-n", "10"])
        .stdin(sort_output)
        .stdout(Stdio::piped())
        .spawn()?
        .wait_with_output()?;

    println!(
        "Top 10 biggest files and directories in '{}':\n{}",
        directory.display(),
        String::from_utf8(head_output.stdout)?
    );

    Ok(())
}
#
# quick_main!(run);

Redirect both stdout and stderr of child process to the same file

std-badge cat-os-badge

Spawns a child process and redirects stdout and stderr to the same file. It follows the same idea as run piped external commands, however process::Stdio will write to the provided files and beforehand, File::try_clone is used to reference the same file handle for stdout and stderr. It will ensure that both handles write with the same cursor position.

The below recipe is equivalent to run the Unix shell command ls . oops >out.txt 2>&1.

# #[macro_use]
# extern crate error_chain;
#
use std::fs::File;
use std::process::{Command, Stdio};

# error_chain! {
#     foreign_links {
#         Io(std::io::Error);
#     }
# }
#
fn run() -> Result<()> {
    let outputs = File::create("out.txt")?;
    let errors = outputs.try_clone()?;

    Command::new("ls")
        .args(&[".", "oops"])
        .stdout(Stdio::from(outputs))
        .stderr(Stdio::from(errors))
        .spawn()?
        .wait_with_output()?;

    Ok(())
}
#
# quick_main!(run);

Continuously process child process' outputs

std-badge cat-os-badge

In Run an external command and process stdout, processing doesn't start until external Command is finished. The recipe below creates a new pipe by calling Stdio::piped and reads stdout continuously as soon as the BufReader is updated.

The below recipe is equivalent to the Unix shell command journalctl | grep usb.

# #[macro_use]
# extern crate error_chain;
#
use std::process::{Command, Stdio};
use std::io::{BufRead, BufReader};

# error_chain! {
#     foreign_links {
#         Io(std::io::Error);
#     }
# }
#
fn run() -> Result<()> {
    let stdout = Command::new("journalctl")
        .stdout(Stdio::piped())
        .spawn()?
        .stdout
        .ok_or_else(|| "Could not capture standard output.")?;

    let reader = BufReader::new(stdout);

    reader
        .lines()
        .filter_map(|line| line.ok())
        .filter(|line| line.find("usb").is_some())
        .for_each(|line| println!("{}", line));

     Ok(())
}
#
# quick_main!(run);

Filter a log file by matching multiple regular expressions

regex-badge cat-text-processing-badge

Reads a file named application.log and only outputs the lines containing “version X.X.X”, some IP address followed by port 443 (e.g. “192.168.0.1:443”), or a specific warning.

A regex::RegexSet is built with regex::RegexSetBuilder. Since backslashes are very common in regular expressions, using raw string literals make them more readable.

# #[macro_use]
# extern crate error_chain;
extern crate regex;

use std::fs::File;
use std::io::{BufReader, BufRead};
use regex::RegexSetBuilder;

# error_chain! {
#     foreign_links {
#         Io(std::io::Error);
#         Regex(regex::Error);
#     }
# }
#
fn run() -> Result<()> {
    let log_path = "application.log";
    let buffered = BufReader::new(File::open(log_path)?);

    let set = RegexSetBuilder::new(&[
        r#"version "\d\.\d\.\d""#,
        r#"\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}:443"#,
        r#"warning.*timeout expired"#,
    ]).case_insensitive(true)
        .build()?;

    buffered
        .lines()
        .filter_map(|line| line.ok())
        .filter(|line| set.is_match(line.as_str()))
        .for_each(|x| println!("{}", x));

    Ok(())
}
#
# quick_main!(run);

Declare lazily evaluated constant

lazy_static-badge cat-caching-badge cat-rust-patterns-badge

Declares a lazily evaluated constant HashMap. The HashMap will be evaluated once and stored behind a global static reference.

#[macro_use]
extern crate lazy_static;

use std::collections::HashMap;

lazy_static! {
    static ref PRIVILEGES: HashMap<&'static str, Vec<&'static str>> = {
        let mut map = HashMap::new();
        map.insert("James", vec!["user", "admin"]);
        map.insert("Jim", vec!["user"]);
        map
    };
}

fn show_access(name: &str) {
    let access = PRIVILEGES.get(name);
    println!("{}: {:?}", name, access);
}

fn main() {
    let access = PRIVILEGES.get("James");
    println!("James: {:?}", access);

    show_access("Jim");
}

Maintain global mutable state

lazy_static-badge cat-rust-patterns-badge

Declares some global state using lazy_static. Since lazy_static creates a globally available static ref we also need to wrap our state in a Mutex to allow mutation (also see RwLock). The Mutex ensures the state cannot be simultaneously accessed by multiple threads, preventing race conditions. A MutexGuard must be acquired to read or mutate the value stored in a Mutex.

# #[macro_use]
# extern crate error_chain;
#[macro_use]
extern crate lazy_static;

use std::sync::Mutex;
#
# error_chain!{ }

lazy_static! {
    static ref FRUIT: Mutex<Vec<String>> = Mutex::new(Vec::new());
}

fn insert(fruit: &str) -> Result<()> {
    // acquire exclusive access
    let mut db = FRUIT.lock().map_err(|_| "Failed to acquire MutexGuard")?;
    db.push(fruit.to_string());
    Ok(())
    // release exclusive access
}

fn run() -> Result<()> {
    insert("apple")?;
    insert("orange")?;
    insert("peach")?;
    {
        // acquire access
        let db = FRUIT.lock().map_err(|_| "Failed to acquire MutexGuard")?;

        db.iter().enumerate().for_each(|(i, item)| println!("{}: {}", i, item));
        // release access
    }
    insert("grape")?;
    Ok(())
}
#
# quick_main!(run);

Verify and extract login from an email address

regex-badge lazy_static-badge cat-text-processing-badge

Validates that an email address is formatted correctly, and extracts everything before the @ symbol.

#[macro_use]
extern crate lazy_static;
extern crate regex;

use regex::Regex;

fn extract_login(input: &str) -> Option<&str> {
    lazy_static! {
        static ref RE: Regex = Regex::new(r"(?x)
            ^(?P<login>[^@\s]+)@
            ([[:word:]]+\.)*
            [[:word:]]+$
            ").unwrap();
    }
    RE.captures(input).and_then(|cap| {
        cap.name("login").map(|login| login.as_str())
    })
}

fn main() {
    assert_eq!(extract_login(r"I❤email@example.com"), Some(r"I❤email"));
    assert_eq!(
        extract_login(r"sdf+sdsfsd.as.sdsd@jhkk.d.rl"),
        Some(r"sdf+sdsfsd.as.sdsd")
    );
    assert_eq!(extract_login(r"More@Than@One@at.com"), None);
    assert_eq!(extract_login(r"Not an email@email"), None);
}

Extract a list of unique #Hashtags from a text

regex-badge lazy_static-badge cat-text-processing-badge

Extracts a sorted and deduplicated list of hashtags from a text.

The hashtag regex given here only catches Latin hashtags that start with a letter. The complete twitter hashtag regex is much more complicated.

extern crate regex;
#[macro_use]
extern crate lazy_static;

use regex::Regex;
use std::collections::HashSet;

/// Note: A HashSet does not contain duplicate values.
fn extract_hashtags(text: &str) -> HashSet<&str> {
    lazy_static! {
        static ref HASHTAG_REGEX : Regex = Regex::new(
                r"\#[a-zA-Z][0-9a-zA-Z_]*"
            ).unwrap();
    }
    HASHTAG_REGEX.find_iter(text).map(|mat| mat.as_str()).collect()
}

fn main() {
    let tweet = "Hey #world, I just got my new #dog, say hello to Till. #dog #forever #2 #_ ";
    let tags = extract_hashtags(tweet);
    assert!(tags.contains("#dog") && tags.contains("#forever") && tags.contains("#world"));
    assert_eq!(tags.len(), 3);
}

Replace all occurrences of one text pattern with another pattern.

regex-badge lazy_static-badge cat-text-processing-badge

Replaces all occurrences of the standard ISO 8601 YYYY-MM-DD date pattern with the equivalent American English date with slashes; for example 2013-01-15 becomes 01/15/2013.

The method Regex::replace_all replaces all occurrences of the whole regex. The Replacer trait helps to figure out the replacement string. This trait is implemented for &str and allows to use variables like $abcde to refer to corresponding named capture groups (?P<abcde>REGEX) from the search regex. See the replacement string syntax for examples and information about escaping.

extern crate regex;
#[macro_use]
extern crate lazy_static;

use std::borrow::Cow;
use regex::Regex;

fn reformat_dates(before: &str) -> Cow<str> {
    lazy_static! {
        static ref ISO8601_DATE_REGEX : Regex = Regex::new(
            r"(?P<y>\d{4})-(?P<m>\d{2})-(?P<d>\d{2})"
            ).unwrap();
    }
    ISO8601_DATE_REGEX.replace_all(before, "$m/$d/$y")
}

fn main() {
    let before = "2012-03-14, 2013-01-15 and 2014-07-05";
    let after = reformat_dates(before);
    assert_eq!(after, "03/14/2012, 01/15/2013 and 07/05/2014");
}

Extract phone numbers from text

regex-badge cat-text-processing-badge

Processes a string of text using Regex::captures_iter to capture multiple phone numbers. The example here is for US convention phone numbers.

# #[macro_use]
# extern crate error_chain;
extern crate regex;

use regex::Regex;
use std::fmt;
#
# error_chain!{
#     foreign_links {
#         Regex(regex::Error);
#         Io(std::io::Error);
#     }
# }

struct PhoneNumber<'a> {
    area: &'a str,
    exchange: &'a str,
    subscriber: &'a str,
}

// Allows printing phone numbers based on country convention.
impl<'a> fmt::Display for PhoneNumber<'a> {
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
        write!(f, "1 ({}) {}-{}", self.area, self.exchange, self.subscriber)
    }
}

fn run() -> Result<()> {
    let phone_text = "
    +1 505 881 9292 (v) +1 505 778 2212 (c) +1 505 881 9297 (f)
    (202) 991 9534
    Alex 5553920011
    1 (800) 233-2010
    1.299.339.1020";

    let re = Regex::new(
        r#"(?x)
          (?:\+?1)?                       # Country Code Optional
          [\s\.]?
          (([2-9]\d{2})|\(([2-9]\d{2})\)) # Area Code
          [\s\.\-]?
          ([2-9]\d{2})                    # Exchange Code
          [\s\.\-]?
          (\d{4})                         # Subscriber Number"#,
    )?;

    let phone_numbers = re.captures_iter(phone_text).filter_map(|cap| {
        // Area code populates either capture group 2 or 3.
        // Group 1 contains optional parenthesis.
        let groups = (cap.get(2).or(cap.get(3)), cap.get(4), cap.get(5));
        match groups {
            (Some(area), Some(ext), Some(sub)) => Some(PhoneNumber {
                area: area.as_str(),
                exchange: ext.as_str(),
                subscriber: sub.as_str(),
            }),
            _ => None,
        }
    });

    assert_eq!(
        phone_numbers.map(|m| m.to_string()).collect::<Vec<_>>(),
        vec![
            "1 (505) 881-9292",
            "1 (505) 778-2212",
            "1 (505) 881-9297",
            "1 (202) 991-9534",
            "1 (555) 392-0011",
            "1 (800) 233-2010",
            "1 (299) 339-1020",
        ]
    );

    Ok(())
}
#
# quick_main!(run);

Calculate the SHA-256 digest of a file

ring-badge data-encoding-badge cat-cryptography-badge

Writes some data to a file, then calculates the SHA-256 digest::Digest of the file's contents using digest::Context.

# #[macro_use]
# extern crate error_chain;
extern crate data_encoding;
extern crate ring;

use data_encoding::HEXUPPER;
use ring::digest::{Context, Digest, SHA256};
use std::fs::File;
use std::io::{BufReader, Read, Write};
#
# error_chain! {
#     foreign_links {
#         Io(std::io::Error);
#         Decode(data_encoding::DecodeError);
#     }
# }

fn sha256_digest<R: Read>(mut reader: R) -> Result<Digest> {
    let mut context = Context::new(&SHA256);
    let mut buffer = [0; 1024];

    loop {
        let count = reader.read(&mut buffer)?;
        if count == 0 {
            break;
        }
        context.update(&buffer[..count]);
    }

    Ok(context.finish())
}

fn run() -> Result<()> {
    let path = "file.txt";

    let mut output = File::create(path)?;
    write!(output, "We will generate a digest of this text")?;

    let input = File::open(path)?;
    let reader = BufReader::new(input);
    let digest = sha256_digest(reader)?;

    // digest.as_ref() provides the digest as a byte slice: &[u8]
    println!("SHA-256 digest is {}", HEXUPPER.encode(digest.as_ref()));

    Ok(())
}
#
# quick_main!(run);

Sign and verify a message with HMAC digest

ring-badge cat-cryptography-badge

Uses ring::hmac to creates a hmac::Signature of a string then verifies the signature is correct.

# #[macro_use]
# extern crate error_chain;
extern crate ring;
#
# error_chain! {
#     foreign_links {
#         Ring(ring::error::Unspecified);
#     }
# }

use ring::{digest, hmac, rand};
use ring::rand::SecureRandom;

fn run() -> Result<()> {
    let mut key_value = [0u8; 48];
    let rng = rand::SystemRandom::new();
    rng.fill(&mut key_value)?;
    let key = hmac::SigningKey::new(&digest::SHA256, &key_value);

    let message = "Legitimate and important message.";
    let signature = hmac::sign(&key, message.as_bytes());
    hmac::verify_with_own_key(&key, message.as_bytes(), signature.as_ref())?;

    Ok(())
}
#
# quick_main!(run);

Salt and hash a password with PBKDF2

ring-badge data-encoding-badge cat-cryptography-badge

Uses ring::pbkdf2 to hash a salted password using the PBKDF2 key derivation function pbkdf2::derive and then verifies the hash is correct with pbkdf2::verify. The salt is generated using SecureRandom::fill, which fills the salt byte array with securely generated random numbers.

# #[macro_use]
# extern crate error_chain;
extern crate data_encoding;
extern crate ring;
#
# error_chain! {
#   foreign_links {
#     Ring(ring::error::Unspecified);
#   }
# }

use data_encoding::HEXUPPER;
use ring::{digest, pbkdf2, rand};
use ring::rand::SecureRandom;

fn run() -> Result<()> {
  const CREDENTIAL_LEN: usize = digest::SHA512_OUTPUT_LEN;
  const N_ITER: u32 = 100_000;
  let rng = rand::SystemRandom::new();

  // Generate salt
  let mut salt = [0u8; CREDENTIAL_LEN];
  rng.fill(&mut salt)?;

  // Hash password
  let password = "Guess Me If You Can!";
  let mut pbkdf2_hash = [0u8; CREDENTIAL_LEN];
  pbkdf2::derive(
      &digest::SHA512,
      N_ITER,
      &salt,
      password.as_bytes(),
      &mut pbkdf2_hash,
  );
  println!("Salt: {}", HEXUPPER.encode(&salt));
  println!("PBKDF2 hash: {}", HEXUPPER.encode(&pbkdf2_hash));

  // Verify hash with correct password and wrong password
  let should_succeed = pbkdf2::verify(
      &digest::SHA512,
      N_ITER,
      &salt,
      password.as_bytes(),
      &pbkdf2_hash,
  );
  let wrong_password = "Definitely not the correct password";
  let should_fail = pbkdf2::verify(
      &digest::SHA512,
      N_ITER,
      &salt,
      wrong_password.as_bytes(),
      &pbkdf2_hash,
  );

  assert!(should_succeed.is_ok());
  assert!(!should_fail.is_ok());

  Ok(())
}
#
# quick_main!(run);

Define and operate on a type represented as a bitfield

bitflags-badge cat-no-std-badge

Creates typesafe bitfield type MyFlags with help of bitflags! macro and implements elementary clear operation as well as Display trait for it. Subsequently, shows basic bitwise operations and formatting.

#[macro_use]
extern crate bitflags;

use std::fmt;

bitflags! {
    struct MyFlags: u32 {
        const FLAG_A       = 0b00000001;
        const FLAG_B       = 0b00000010;
        const FLAG_C       = 0b00000100;
        const FLAG_ABC     = Self::FLAG_A.bits
                           | Self::FLAG_B.bits
                           | Self::FLAG_C.bits;
    }
}

impl MyFlags {
    pub fn clear(&mut self) -> &mut MyFlags {
        self.bits = 0;  // The `bits` field can be accessed from within the
                        // same module where the `bitflags!` macro was invoked.
        self
    }
}

impl fmt::Display for MyFlags {
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
        write!(f, "{:032b}", self.bits)
    }
}

fn main() {
    let e1 = MyFlags::FLAG_A | MyFlags::FLAG_C;
    let e2 = MyFlags::FLAG_B | MyFlags::FLAG_C;
    assert_eq!((e1 | e2), MyFlags::FLAG_ABC);   // union
    assert_eq!((e1 & e2), MyFlags::FLAG_C);     // intersection
    assert_eq!((e1 - e2), MyFlags::FLAG_A);     // set difference
    assert_eq!(!e2, MyFlags::FLAG_A);           // set complement

    let mut flags = MyFlags::FLAG_ABC;
    assert_eq!(format!("{}", flags), "00000000000000000000000000000111");
    assert_eq!(format!("{}", flags.clear()), "00000000000000000000000000000000");
    // Debug trait is automatically derived for the MyFlags through `bitflags!`
    assert_eq!(format!("{:?}", MyFlags::FLAG_B), "FLAG_B");
    assert_eq!(format!("{:?}", MyFlags::FLAG_A | MyFlags::FLAG_B), "FLAG_A | FLAG_B");
}

Access a file randomly using a memory map

memmap-badge cat-filesystem-badge

Creates a memory map of a file using memmap and simulates some non-sequential reads from the file. Using a memory map means you just index into a slice rather than dealing with seeking around in a File.

The Mmap::map function is only safe if we can guarantee that the file behind the memory map is not being modified at the same time by another process, as this would be a race condition.

# #[macro_use]
# extern crate error_chain;
extern crate memmap;

use memmap::Mmap;
# use std::fs::File;
# use std::io::Write;
#
# error_chain! {
#     foreign_links {
#         Io(std::io::Error);
#     }
# }

fn run() -> Result<()> {
#     write!(File::create("content.txt")?, "My hovercraft is full of eels!")?;
#
    let file = File::open("content.txt")?;
    let map = unsafe { Mmap::map(&file)? };

    let random_indexes = [0, 1, 2, 19, 22, 10, 11, 29];
    assert_eq!(&map[3..13], b"hovercraft");
    // I'm using an iterator here to change indexes to bytes
    let random_bytes: Vec<u8> = random_indexes.iter()
        .map(|&idx| map[idx])
        .collect();
    assert_eq!(&random_bytes[..], b"My loaf!");
    Ok(())
}
#
# quick_main!(run);

Check number of logical cpu cores

num_cpus-badge cat-hardware-support-badge

Shows the number of logical cpu cores in current machine using num_cpus::get.

extern crate num_cpus;

fn main() {
    println!("Number of logical cores is {}", num_cpus::get());
}

Handle errors correctly in main

error-chain-badge cat-rust-patterns-badge

Handles error that occur when trying to open a file that does not exist. It is achieved by using error-chain, a library that takes care of a lot of boilerplate code needed in order to handle errors in Rust.

Io(std::io::Error) inside foreign_links allows automatic conversion from std::io::Error into error_chain! defined type implementing the Error trait.

The below recipe will tell how long the system has been running by opening the Unix file /proc/uptime and parse the content to get the first number. Returns uptime unless there is an error.

#[macro_use]
extern crate error_chain;

use std::fs::File;
use std::io::Read;

error_chain!{
    foreign_links {
        Io(std::io::Error);
        ParseInt(::std::num::ParseIntError);
    }
}

fn read_uptime() -> Result<u64> {
    let mut uptime = String::new();
    File::open("/proc/uptime")?.read_to_string(&mut uptime)?;

    Ok(uptime
        .split('.')
        .next()
        .ok_or("Cannot parse uptime data")?
        .parse()?)
}

fn main() {
    match read_uptime() {
        Ok(uptime) => println!("uptime: {} seconds", uptime),
        Err(err) => eprintln!("error: {}", err),
    };
}

Avoid discarding errors during error conversions

error-chain-badge cat-rust-patterns-badge

Matching on different error types returned by a function is possible and relatively compact with error-chain crate. To do so, ErrorKind is used to determine the error type.

To illustrate the error matching, a random integer generator web service will be queried via reqwest and then the response will be parsed. Errors can be generated by the Rust standard library, reqwest and by the web service. foreign_links are used for well defined Rust errors. The errors block of the error_chain! macro is used to create an additional ErrorKind variant for the web service error. Finally, a regular match can be used to react differently according to the raised error.

#[macro_use]
extern crate error_chain;
extern crate reqwest;

use std::io::Read;

error_chain! {
    foreign_links {
        Io(std::io::Error);
        Reqwest(reqwest::Error);
        ParseIntError(std::num::ParseIntError);
    }

    errors { RandomResponseError(t: String) }
}

fn parse_response(mut response: reqwest::Response) -> Result<u32> {
    let mut body = String::new();
    response.read_to_string(&mut body)?;
    body.pop();
    body.parse::<u32>()
        .chain_err(|| ErrorKind::RandomResponseError(body))
}

fn run() -> Result<()> {
    let url =
        format!("https://www.random.org/integers/?num=1&min=0&max=10&col=1&base=10&format=plain");
    let response = reqwest::get(&url)?;
    let random_value: u32 = parse_response(response)?;

    println!("a random number between 0 and 10: {}", random_value);

    Ok(())
}

fn main() {
    if let Err(error) = run() {
        match *error.kind() {
            ErrorKind::Io(_) => println!("Standard IO error: {:?}", error),
            ErrorKind::Reqwest(_) => println!("Reqwest error: {:?}", error),
            ErrorKind::ParseIntError(_) => println!("Standard parse int error: {:?}", error),
            ErrorKind::RandomResponseError(_) => println!("User defined error: {:?}", error),
            _ => println!("Other error: {:?}", error),
        }
    }
}

Obtain backtrace of complex error scenarios

error-chain-badge cat-rust-patterns-badge

This recipe shows how to handle a complex error scenario and then print a backtrace. It relies on chain_err to extend errors by appending new errors. The error stack can be unwinded, thus providing a better context to understand why an error was raised.

The below recipes attempts to deserialize the value 256 into a u8. An error will bubble up from Serde then csv and finally up to the user code.

# extern crate csv;
#[macro_use]
extern crate error_chain;
# #[macro_use]
# extern crate serde_derive;
#
# use std::fmt;
#
# error_chain! {
#     foreign_links {
#         Reader(csv::Error);
#     }
# }

#[derive(Debug, Deserialize)]
struct Rgb {
    red: u8,
    blue: u8,
    green: u8,
}

impl Rgb {
    fn from_reader(csv_data: &[u8]) -> Result<Rgb> {
        let color: Rgb = csv::Reader::from_reader(csv_data)
            .deserialize()
            .nth(0)
            .ok_or("Cannot deserialize the first CSV record")?
            .chain_err(|| "Cannot deserialize RGB color")?;

        Ok(color)
    }
}

# impl fmt::UpperHex for Rgb {
#     fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
#         let hexa = u32::from(self.red) << 16 | u32::from(self.blue) << 8 | u32::from(self.green);
#         write!(f, "{:X}", hexa)
#     }
# }
#
fn run() -> Result<()> {
    let csv = "red,blue,green
102,256,204";

    let rgb = Rgb::from_reader(csv.as_bytes()).chain_err(|| "Cannot read CSV data")?;
    println!("{:?} to hexadecimal #{:X}", rgb, rgb);

    Ok(())
}

fn main() {
    if let Err(ref errors) = run() {
        eprintln!("Error level - description");
        errors
            .iter()
            .enumerate()
            .for_each(|(index, error)| eprintln!("└> {} - {}", index, error));

        if let Some(backtrace) = errors.backtrace() {
            eprintln!("{:?}", backtrace);
        }
#
#         // In a real use case, errors should handled. For example:
#         // ::std::process::exit(1);
    }
}

Backtrace error rendered:

Error level - description
└> 0 - Cannot read CSV data
└> 1 - Cannot deserialize RGB color
└> 2 - CSV deserialize error: record 1 (line: 2, byte: 15): field 1: number too large to fit in target type
└> 3 - field 1: number too large to fit in target type

Run the recipe with RUST_BACKTRACE=1 to display a detailed backtrace associated with this error.

Measure the elapsed time between two code sections

std-badge cat-time-badge

Measures time::Instant::elapsed since time::Instant::now.

Calling time::Instant::elapsed returns a time::Duration that we print at the end of the example. This method will not mutate or reset the time::Instant object.

use std::time::{Duration, Instant};
# use std::thread;
#
# fn expensive_function() {
#     thread::sleep(Duration::from_secs(1));
# }

fn main() {
    let start = Instant::now();
    expensive_function();
    let duration = start.elapsed();

    println!("Time elapsed in expensive_function() is: {:?}", duration);
}

Convert a local time to an another UTC timezone and vice versa

chrono-badge cat-date-and-time-badge

Gets the local time and displays it using offset::Local::now and then converts it to the UTC standard using the DateTime::from_utc struct method. A time is then converted using the offset::FixedOffset struct and the UTC time is then converted to UTC+8 and UTC-2.

extern crate chrono;

use chrono::{DateTime, FixedOffset, Local, Utc};

fn main() {
    let local_time = Local::now();
    let utc_time = DateTime::<Utc>::from_utc(local_time.naive_utc(), Utc);
    let china_timezone = FixedOffset::east(8 * 3600);
    let rio_timezone = FixedOffset::west(2 * 3600);
    println!("Local time now is {}", local_time);
    println!("UTC time now is {}", utc_time);
    println!(
        "Time in Hong Kong now is {}",
        utc_time.with_timezone(&china_timezone)
    );
    println!("Time in Rio de Janeiro now is {}", utc_time.with_timezone(&rio_timezone));
}

Convert date to UNIX timestamp and vice versa

chrono-badge cat-date-and-time-badge

Converts a date given by NaiveDate::from_ymd and NaiveTime::from_hms to UNIX timestamp using NaiveDateTime::timestamp. Then it calculates what was the date after one billion seconds since January 1, 1970 0:00:00 UTC, using NaiveDateTime::from_timestamp.

extern crate chrono;

use chrono::{NaiveDate, NaiveDateTime};

fn main() {
    let date_time: NaiveDateTime = NaiveDate::from_ymd(2017, 11, 12).and_hms(17, 33, 44);
    println!(
        "Number of seconds between 1970-01-01 00:00:00 and {} is {}.",
        date_time, date_time.timestamp());

    let date_time_after_a_billion_seconds = NaiveDateTime::from_timestamp(1_000_000_000, 0);
    println!(
        "Date after a billion seconds since 1970-01-01 00:00:00 was {}.",
        date_time_after_a_billion_seconds);
}

Display formatted date and time

chrono-badge cat-date-and-time-badge

Gets and displays the current time in UTC using Utc::now. Formats the current time in the well-known formats RFC 2822 using DateTime::to_rfc2822 and RFC 3339 using DateTime::to_rfc3339, and in a custom format using DateTime::format.

extern crate chrono;
use chrono::{DateTime, Utc};

fn main() {
    let now: DateTime<Utc> = Utc::now();

    println!("UTC now is: {}", now);
    println!("UTC now in RFC 2822 is: {}", now.to_rfc2822());
    println!("UTC now in RFC 3339 is: {}", now.to_rfc3339());
    println!("UTC now in a custom format is: {}", now.format("%a %b %e %T %Y"));
}

Parse string into DateTime struct

chrono-badge cat-date-and-time-badge

Parses a DateTime struct from strings representing the well-known formats RFC 2822, RFC 3339, and a custom format, using DateTime::parse_from_rfc2822, DateTime::parse_from_rfc3339, and DateTime::parse_from_str respectively.

Escape sequences that are available for the DateTime::parse_from_str can be found at chrono::format::strftime. Note that the DateTime::parse_from_str requires that such a DateTime struct can be created that it uniquely identifies a date and a time. For parsing dates and times without timezones use NaiveDate, NaiveTime, and NaiveDateTime.

extern crate chrono;
# #[macro_use]
# extern crate error_chain;
#
use chrono::{DateTime, NaiveDate, NaiveDateTime, NaiveTime};
#
# error_chain! {
#     foreign_links {
#         DateParse(chrono::format::ParseError);
#     }
# }

fn run() -> Result<()> {
    let rfc2822 = DateTime::parse_from_rfc2822("Tue, 1 Jul 2003 10:52:37 +0200")?;
    println!("{}", rfc2822);

    let rfc3339 = DateTime::parse_from_rfc3339("1996-12-19T16:39:57-08:00")?;
    println!("{}", rfc3339);

    let custom = DateTime::parse_from_str("5.8.1994 8:00 am +0000", "%d.%m.%Y %H:%M %P %z")?;
    println!("{}", custom);

    let time_only = NaiveTime::parse_from_str("23:56:04", "%H:%M:%S")?;
    println!("{}", time_only);

    let date_only = NaiveDate::parse_from_str("2015-09-05", "%Y-%m-%d")?;
    println!("{}", date_only);

    let no_timezone = NaiveDateTime::parse_from_str("2015-09-05 23:56:04", "%Y-%m-%d %H:%M:%S")?;
    println!("{}", no_timezone);

    Ok(())
}
#
# quick_main!(run);

Perform checked date and time calculations

chrono-badge cat-date-and-time-badge

Calculates and displays the date and time two weeks from now using DateTime::checked_add_signed and the date of the day before that using DateTime::checked_sub_signed. The methods return None if the date and time cannot be calculated.

Escape sequences that are available for the DateTime::format can be found at chrono::format::strftime.

extern crate chrono;
use chrono::{DateTime, Duration, Utc};

fn day_earlier(date_time: DateTime<Utc>) -> Option<DateTime<Utc>> {
    date_time.checked_sub_signed(Duration::days(1))
}

fn main() {
    let now = Utc::now();
    println!("{}", now);

    let almost_three_weeks_from_now = now.checked_add_signed(Duration::weeks(2))
            .and_then(|in_2weeks| in_2weeks.checked_add_signed(Duration::weeks(1)))
            .and_then(day_earlier);

    match almost_three_weeks_from_now {
        Some(x) => println!("{}", x),
        None => eprintln!("Almost three weeks from now overflows!"),
    }

    match now.checked_add_signed(Duration::max_value()) {
        Some(x) => println!("{}", x),
        None => eprintln!("We can't use chrono to tell the time for the Solar System to complete more than one full orbit around the galactic center."),
    }
}

Examine the date and time

chrono-badge cat-date-and-time-badge

Gets the current UTC DateTime and its hour/minute/second via Timelike and its year/month/day/weekday via Datelike.

extern crate chrono;
use chrono::{Datelike, Timelike, Utc};

fn main() {
    let now = Utc::now();

    let (is_pm, hour) = now.hour12();
    println!(
        "The current UTC time is {:02}:{:02}:{:02} {}",
        hour,
        now.minute(),
        now.second(),
        if is_pm { "PM" } else { "AM" }
    );
    println!(
        "And there have been {} seconds since midnight",
        now.num_seconds_from_midnight()
    );

    let (is_common_era, year) = now.year_ce();
    println!(
        "The current UTC date is {}-{:02}-{:02} {:?} ({})",
        year,
        now.month(),
        now.day(),
        now.weekday(),
        if is_common_era { "CE" } else { "BCE" }
    );
    println!(
        "And the Common Era began {} days ago",
        now.num_days_from_ce()
    );
}

File names that have been modified in the last 24 hours for the working directory

std-badge cat-filesystem-badge

Gets the current working directory by calling env::current_dir, then for each entries in fs::read_dir, extracts the DirEntry::path and gets the metadata via fs::Metadata. The Metadata::modified returns the SystemTime::elapsed time since last modification of the entry. It's converted into seconds with Duration::as_secs and compared with 24 hours (24 * 60 * 60 seconds). Metadata::is_file is used to filter out directories.

# #[macro_use]
# extern crate error_chain;
#
use std::{env, fs};

# error_chain! {
#     foreign_links {
#         Io(std::io::Error);
#         SystemTimeError(std::time::SystemTimeError);
#     }
# }
#
fn run() -> Result<()> {
    let current_dir = env::current_dir()?;
    println!(
        "Entries modified in the last 24 hours in {:?}:",
        current_dir
    );

    for entry in fs::read_dir(current_dir)? {
        let entry = entry?;
        let path = entry.path();

        let metadata = fs::metadata(&path)?;
        let last_modified = metadata.modified()?.elapsed()?.as_secs();

        if last_modified < 24 * 3600 && metadata.is_file() {
            println!(
                "Last modified: {:?} seconds, is read only: {:?}, size: {:?} bytes, filename: {:?}",
                last_modified,
                metadata.permissions().readonly(),
                metadata.len(),
                path.file_name().ok_or("No filename")?
            );
        }
    }

    Ok(())
}
#
# quick_main!(run);

Encoding

Recipe Crates Categories
Serialize and deserialize unstructured JSON serde-json-badge cat-encoding-badge
Deserialize a TOML configuration file toml-badge cat-encoding-badge
Percent-encode a string url-badge cat-encoding-badge
Encode a string as application/x-www-form-urlencoded url-badge cat-encoding-badge
Encode and decode hex data-encoding-badge cat-encoding-badge
Encode and decode base64 base64-badge cat-encoding-badge
Read CSV records csv-badge cat-encoding-badge
Read CSV records with different delimiter csv-badge cat-encoding-badge
Filter CSV records matching a predicate csv-badge cat-encoding-badge
Handle invalid CSV data with Serde csv-badge serde-badge cat-encoding-badge
Serialize records to CSV csv-badge cat-encoding-badge
Serialize records to CSV using Serde csv-badge serde-badge cat-encoding-badge
Transform one column of a CSV file csv-badge serde-badge cat-encoding-badge
Get MIME type from string mime-badge cat-encoding-badge

Serialize and deserialize unstructured JSON

serde-json-badge cat-encoding-badge

The [serde_json] crate provides a serde_json::from_str function to parse a &str of JSON into a type of the caller's choice.

Unstructured JSON can be parsed into a universal serde_json::Value type that is able to represent any valid JSON data.

The example below shows a &str of JSON being parsed and then compared to what we expect the parsed value to be. The expected value is declared using the json! macro.

# #[macro_use]
# extern crate error_chain;
#[macro_use]
extern crate serde_json;

use serde_json::Value;
#
# error_chain! {
#     foreign_links {
#         Json(serde_json::Error);
#     }
# }

fn run() -> Result<()> {
    let j = r#"{
                 "userid": 103609,
                 "verified": true,
                 "access_privileges": [
                   "user",
                   "admin"
                 ]
               }"#;

    let parsed: Value = serde_json::from_str(j)?;

    let expected = json!({
        "userid": 103609,
        "verified": true,
        "access_privileges": [
            "user",
            "admin"
        ]
    });

    assert_eq!(parsed, expected);

    Ok(())
}
#
# quick_main!(run);

Deserialize a TOML configuration file

toml-badge cat-encoding-badge

Parse some TOML into a universal toml::Value that is able to represent any valid TOML data.

# #[macro_use]
# extern crate error_chain;
extern crate toml;

use toml::Value;
#
# error_chain! {
#     foreign_links {
#         Toml(toml::de::Error);
#     }
# }

fn run() -> Result<()> {
    let toml_content = r#"
          [package]
          name = "your_package"
          version = "0.1.0"
          authors = ["You! <you@example.org>"]

          [dependencies]
          serde = "1.0"
          "#;

    let package_info: Value = toml::from_str(toml_content)?;

    assert_eq!(package_info["dependencies"]["serde"].as_str(), Some("1.0"));
    assert_eq!(package_info["package"]["name"].as_str(),
               Some("your_package"));

    Ok(())
}
#
# quick_main!(run);

Parse TOML into your own structs using Serde:

serde-json-badge toml-badge cat-encoding-badge

# #[macro_use]
# extern crate error_chain;
#[macro_use]
extern crate serde_derive;
extern crate toml;

use std::collections::HashMap;

#[derive(Deserialize)]
struct Config {
    package: Package,
    dependencies: HashMap<String, String>,
}

#[derive(Deserialize)]
struct Package {
    name: String,
    version: String,
    authors: Vec<String>,
}
#
# error_chain! {
#     foreign_links {
#         Toml(toml::de::Error);
#     }
# }

fn run() -> Result<()> {
    let toml_content = r#"
          [package]
          name = "your_package"
          version = "0.1.0"
          authors = ["You! <you@example.org>"]

          [dependencies]
          serde = "1.0"
          "#;

    let package_info: Config = toml::from_str(toml_content)?;

    assert_eq!(package_info.package.name, "your_package");
    assert_eq!(package_info.package.version, "0.1.0");
    assert_eq!(package_info.package.authors, vec!["You! <you@example.org>"]);
    assert_eq!(package_info.dependencies["serde"], "1.0");

    Ok(())
}
#
# quick_main!(run);

Percent-encode a string

url-badge cat-encoding-badge

Encode an input string with percent-encoding using the utf8_percent_encode function from the url crate. Then decode using the percent_decode function.

# #[macro_use]
# extern crate error_chain;
extern crate url;

use url::percent_encoding::{utf8_percent_encode, percent_decode, DEFAULT_ENCODE_SET};
#
# error_chain! {
#     foreign_links {
#         Utf8(std::str::Utf8Error);
#     }
# }

fn run() -> Result<()> {
    let input = "confident, productive systems programming";

    let iter = utf8_percent_encode(input, DEFAULT_ENCODE_SET);
    let encoded: String = iter.collect();
    assert_eq!(encoded, "confident,%20productive%20systems%20programming");

    let iter = percent_decode(encoded.as_bytes());
    let decoded = iter.decode_utf8()?;
    assert_eq!(decoded, "confident, productive systems programming");

    Ok(())
}
#
# quick_main!(run);

The encode set defines which bytes (in addition to non-ASCII and controls) need to be percent-encoded. The choice of this set depends on context. For example, ? needs to be encoded in a URL path but not in a query string.

The return value of encoding is an iterator of &str slices which can be collected into a String.

Encode a string as application/x-www-form-urlencoded

url-badge cat-encoding-badge

Encodes a string into application/x-www-form-urlencoded syntax using the form_urlencoded::byte_serialize and subsequently decodes it with form_urlencoded::parse. Both functions return iterators that can be collected into a String.

extern crate url;
use url::form_urlencoded::{byte_serialize, parse};

fn main() {
    let urlencoded: String = byte_serialize("What is ❤?".as_bytes()).collect();
    assert_eq!(urlencoded, "What+is+%E2%9D%A4%3F");
    println!("urlencoded:'{}'", urlencoded);

    let decoded: String = parse(urlencoded.as_bytes())
        .map(|(key, val)| [key, val].concat())
        .collect();
    assert_eq!(decoded, "What is ❤?");
    println!("decoded:'{}'", decoded);
}

Encode and decode hex

data-encoding-badge cat-encoding-badge

The data_encoding crate provides a HEXUPPER::encode method which takes a &[u8] and returns a String containing the hexadecimal representation of the data.

Similarly, a HEXUPPER::decode method is provided which takes a &[u8] and returns a Vec<u8> if the input data is successfully decoded.

The example below shows a &[u8] of data being converted to its hexadecimal representation and then being compared to its expected value. The returned hex String is then converted back to its original representation and is compared to the original value provided.

# #[macro_use]
# extern crate error_chain;
extern crate data_encoding;

use data_encoding::{HEXUPPER, DecodeError};
#
# error_chain! {
#     foreign_links {
#         Decode(DecodeError);
#     }
# }

fn run() -> Result<()> {
    let original = b"The quick brown fox jumps over the lazy dog.";
    let expected = "54686520717569636B2062726F776E20666F78206A756D7073206F76\
        657220746865206C617A7920646F672E";

    let encoded = HEXUPPER.encode(original);
    assert_eq!(encoded, expected);

    let decoded = HEXUPPER.decode(&encoded.into_bytes())?;
    assert_eq!(&decoded[..], &original[..]);

    Ok(())
}
#
# quick_main!(run);

Encode and decode base64

base64-badge cat-encoding-badge

Encodes byte slice into base64 String with help of encode and subsequently decodes it with decode.

# #[macro_use]
# extern crate error_chain;
extern crate base64;

use std::str;
use base64::{encode, decode};
#
# error_chain! {
#     foreign_links {
#         Base64(base64::DecodeError);
#         Utf8Error(str::Utf8Error);
#     }
# }

fn run() -> Result<()> {
    let hello = b"hello rustaceans";
    let encoded = encode(hello);
    let decoded = decode(&encoded)?;

    println!("origin: {}", str::from_utf8(hello)?);
    println!("base64 encoded: {}", encoded);
    println!("back to origin: {}", str::from_utf8(&decoded)?);

    Ok(())
}
#
# quick_main!(run);

Read CSV records

csv-badge cat-encoding-badge

Reads standard CSV records into csv::StringRecord — a weakly typed data representation. It expects to read valid UTF-8 rows. On the other hand, if invalid UTF-8 data has to be read, then prefer using csv::ByteRecord, since it makes no assumptions about UTF-8.

extern crate csv;
# #[macro_use]
# extern crate error_chain;
#
# error_chain! {
#     foreign_links {
#         Reader(csv::Error);
#     }
# }

fn run() -> Result<()> {
    let csv = "year,make,model,description
1948,Porsche,356,Luxury sports car
1967,Ford,Mustang fastback 1967,American car";

    let mut reader = csv::Reader::from_reader(csv.as_bytes());
    for record in reader.records() {
        let record = record?;
        println!(
            "In {}, {} built the {} model. It is a {}.",
            &record[0],
            &record[1],
            &record[2],
            &record[3]
        );
    }

    Ok(())
}
#
# quick_main!(run);

This is like the previous example, however Serde is used to deserialize data into strongly type structures. See the csv::Reader::deserialize method.

extern crate csv;
# #[macro_use]
# extern crate error_chain;
#[macro_use]
extern crate serde_derive;

# error_chain! {
#     foreign_links {
#         Reader(csv::Error);
#     }
# }
#
#[derive(Deserialize)]
struct Record {
    year: u16,
    make: String,
    model: String,
    description: String,
}

fn run() -> Result<()> {
    let csv = "year,make,model,description
1948,Porsche,356,Luxury sports car
1967,Ford,Mustang fastback 1967,American car";

    let mut reader = csv::Reader::from_reader(csv.as_bytes());

    for record in reader.deserialize() {
        let record: Record = record?;
        println!(
            "In {}, {} built the {} model. It is a {}.",
            record.year,
            record.make,
            record.model,
            record.description
        );
    }

    Ok(())
}
#
# quick_main!(run);

Read CSV records with different delimiter

csv-badge cat-encoding-badge

Reads CSV records with delimiter other than ','

# #[macro_use]
# extern crate error_chain;
extern crate csv;
#[macro_use]
extern crate serde_derive;

#[derive(Debug, Deserialize)]
struct Record {
    name: String,
    place: String,
    #[serde(deserialize_with = "csv::invalid_option")]
    id: Option<u64>,
}

use csv::ReaderBuilder;
#
# error_chain! {
#     foreign_links {
#         CsvError(csv::Error);
#     }
# }

fn run() -> Result<()> {
    let data = "name-place-id
Mark-Melbourne-46
Ashley-Zurich-92";

    let mut reader = ReaderBuilder::new().delimiter(b'-').from_reader(data.as_bytes());
    for result in reader.records() {
        println!("{:?}", result?);
    }

    Ok(())
}
#
# quick_main!(run);

Filter CSV records matching a predicate

csv-badge cat-encoding-badge

Returns only the rows from data with a field that matches query.

# #[macro_use]
# extern crate error_chain;
extern crate csv;

use std::io;
#
# error_chain!{
#     foreign_links {
#         Io(std::io::Error);
#         CsvError(csv::Error); // or just Seek(csv::Error)
#     }
# }

fn run() -> Result<()> {
    let query = "CA";
    let data = "\
City,State,Population,Latitude,Longitude
Kenai,AK,7610,60.5544444,-151.2583333
Oakman,AL,,33.7133333,-87.3886111
Sandfort,AL,,32.3380556,-85.2233333
West Hollywood,CA,37031,34.0900000,-118.3608333";

    let mut rdr = csv::ReaderBuilder::new().from_reader(data.as_bytes());
    let mut wtr = csv::Writer::from_writer(io::stdout());

    wtr.write_record(rdr.headers()?)?;

    for result in rdr.records() {
        let record = result?;
        if record.iter().any(|field| field == query) {
            wtr.write_record(&record)?;
        }
    }

    wtr.flush()?;
    Ok(())
}
#
# quick_main!(run);

Disclaimer: this example has been adapted from the csv crate tutorial.

Handle invalid CSV data with Serde

csv-badge serde-badge cat-encoding-badge

CSV files often contain invalid data. For these cases, the csv crate provides a custom deserializer, csv::invalid_option, which automatically converts invalid data to None values.

# #[macro_use]
# extern crate error_chain;
extern crate csv;
#[macro_use]
extern crate serde_derive;

#[derive(Debug, Deserialize)]
struct Record {
    name: String,
    place: String,
    #[serde(deserialize_with = "csv::invalid_option")]
    id: Option<u64>,
}
#
# error_chain! {
#     foreign_links {
#         CsvError(csv::Error);
#     }
# }

fn run() -> Result<()> {
    let data = "name,place,id
mark,sydney,46.5
ashley,zurich,92
akshat,delhi,37
alisha,colombo,xyz";

    let mut rdr = csv::Reader::from_reader(data.as_bytes());
    for result in rdr.deserialize() {
        let record: Record = result?;
        println!("{:?}", record);
    }

    Ok(())
}
#
# quick_main!(run);

Serialize records to CSV

csv-badge cat-encoding-badge

This example shows how to serialize a Rust tuple. csv::writer supports automatic serialization from Rust types into CSV records. write_record is used when writing a simple record that contains string-like data only, serialize is used when data consists of more complex values like numbers, floats or optional values. Since CSV writer uses internal buffer, always explicitly flush when done.

# #[macro_use]
# extern crate error_chain;
extern crate csv;

use std::io;
#
# error_chain! {
#     foreign_links {
#         CSVError(csv::Error);
#         IOError(std::io::Error);
#    }
# }

fn run() -> Result<()> {
    let mut wtr = csv::Writer::from_writer(io::stdout());

    wtr.write_record(&["Name", "Place", "ID"])?;

    wtr.serialize(("Mark", "Sydney", 87))?;
    wtr.serialize(("Ashley", "Dublin", 32))?;
    wtr.serialize(("Akshat", "Delhi", 11))?;

    wtr.flush()?;
    Ok(())
}
#
# quick_main!(run);

Serialize records to CSV using Serde

csv-badge serde-badge cat-encoding-badge

The following example shows how to serialize custom structs as CSV records using the serde crate.

# #[macro_use]
# extern crate error_chain;
extern crate csv;
#[macro_use]
extern crate serde_derive;

use std::io;
#
# error_chain! {
#    foreign_links {
#        IOError(std::io::Error);
#        CSVError(csv::Error);
#    }
# }

#[derive(Serialize)]
struct Record<'a> {
    name: &'a str,
    place: &'a str,
    id: u64,
}

fn run() -> Result<()> {
    let mut wtr = csv::Writer::from_writer(io::stdout());

    let rec1 = Record { name: "Mark", place: "Melbourne", id: 56};
    let rec2 = Record { name: "Ashley", place: "Sydney", id: 64};
    let rec3 = Record { name: "Akshat", place: "Delhi", id: 98};

    wtr.serialize(rec1)?;
    wtr.serialize(rec2)?;
    wtr.serialize(rec3)?;

    wtr.flush()?;

    Ok(())
}
#
# quick_main!(run);

Transform CSV column

csv-badge serde-badge cat-encoding-badge

Transform a CSV file containing a color name and a hex color into one with a color name and an rgb color. Utilizes the csv crate to read and write the csv file, and serde to deserialize and serialize the rows to and from bytes.

See csv::Reader::deserialize, serde::Deserialize, and std::str::FromStr

extern crate csv;
# #[macro_use]
# extern crate error_chain;
#[macro_use]
extern crate serde_derive;
extern crate serde;

use csv::{Reader, Writer};
use serde::{de, Deserialize, Deserializer};
use std::str::FromStr;
#
# error_chain! {
#    foreign_links {
#        CsvError(csv::Error);
#        ParseInt(std::num::ParseIntError);
#        CsvInnerError(csv::IntoInnerError<Writer<Vec<u8>>>);
#        IO(std::fmt::Error);
#        UTF8(std::string::FromUtf8Error);
#    }
# }

#[derive(Debug)]
struct HexColor {
    red: u8,
    green: u8,
    blue: u8,
}

#[derive(Debug, Deserialize)]
struct Row {
    color_name: String,
    color: HexColor,
}

impl FromStr for HexColor {
    type Err = Error;

    fn from_str(hex_color: &str) -> std::result::Result<Self, Self::Err> {
        let trimmed = hex_color.trim_matches('#');
        if trimmed.len() != 6 {
            Err("Invalid length of hex string".into())
        } else {
            Ok(HexColor {
                red: u8::from_str_radix(&trimmed[..2], 16)?,
                green: u8::from_str_radix(&trimmed[2..4], 16)?,
                blue: u8::from_str_radix(&trimmed[4..6], 16)?,
            })
        }
    }
}

impl<'de> Deserialize<'de> for HexColor {
    fn deserialize<D>(deserializer: D) -> std::result::Result<Self, D::Error>
    where
        D: Deserializer<'de>,
    {
        let s = String::deserialize(deserializer)?;
        FromStr::from_str(&s).map_err(de::Error::custom)
    }
}

fn run() -> Result<()> {
    let data = "color_name,color
red,#ff0000
green,#00ff00
blue,#0000FF
periwinkle,#ccccff
magenta,#ff00ff"
        .to_owned();
    let mut out = Writer::from_writer(vec![]);
    let mut reader = Reader::from_reader(data.as_bytes());
    for result in reader.deserialize::<Row>() {
        let res = result?;
        out.serialize((
            res.color_name,
            res.color.red,
            res.color.green,
            res.color.blue,
        ))?;
    }
    let written = String::from_utf8(out.into_inner()?)?;
    assert_eq!(Some("magenta,255,0,255"), written.lines().last());
    println!("{}", written);
    Ok(())
}
#
# quick_main!(run);

Get MIME type from string

mime-badge cat-encoding-badge

The following example shows how to parse a MIME type from a string using the mime crate. You can handle a possible FromStrError by providing a default MIME type in an unwrap_or clause.

extern crate mime;
use mime::{Mime, APPLICATION_OCTET_STREAM};

fn main() {
    let invalid_mime_type = "i n v a l i d";
    let default_mime = invalid_mime_type
        .parse::<Mime>()
        .unwrap_or(APPLICATION_OCTET_STREAM);

    println!(
        "MIME for {:?} used default value {:?}",
        invalid_mime_type, default_mime
    );

    let valid_mime_type = "TEXT/PLAIN";
    let parsed_mime = valid_mime_type
        .parse::<Mime>()
        .unwrap_or(APPLICATION_OCTET_STREAM);

    println!(
        "MIME for {:?} was parsed as {:?}",
        valid_mime_type, parsed_mime
    );
}

Concurrency

Recipe Crates Categories
Mutate the elements of an array in parallel rayon-badge cat-concurrency-badge
Test in parallel if any or all elements of a collection match a given predicate rayon-badge cat-concurrency-badge
Search items using given predicate in parallel rayon-badge cat-concurrency-badge
Sort a vector in parallel rayon-badge rand-badge cat-concurrency-badge
Map-reduce in parallel rayon-badge cat-concurrency-badge
Generate jpg thumbnails in parallel rayon-badge glob-badge image-badge cat-concurrency-badgecat-filesystem-badge
Spawn a short-lived thread crossbeam-badge cat-concurrency-badge
Draw fractal dispatching work to a thread pool threadpool-badge num-badge num_cpus-badge image-badge cat-concurrency-badgecat-science-badgecat-rendering-badge
Calculate SHA1 sum of *.iso files concurrently threadpool-badge walkdir-badge num_cpus-badge ring-badge cat-concurrency-badgecat-filesystem-badge

Mutate the elements of an array in parallel

rayon-badge cat-concurrency-badge

The example uses the rayon crate, which is a data parallelism library for Rust. rayon provides the par_iter_mut method for any parallel iterable data type. It lets us write iterator-like chains that execute in parallel.

extern crate rayon;

use rayon::prelude::*;

fn main() {
    let mut arr = [0, 7, 9, 11];

    arr.par_iter_mut().for_each(|p| *p -= 1);

    println!("{:?}", arr);
}

Test in parallel if any or all elements of a collection match a given predicate

rayon-badge cat-concurrency-badge

This example demonstrates using the rayon::any and rayon::all methods, which are parallelized counterparts to std::any and std::all. rayon::any checks in parallel whether any element of the iterator matches the predicate, and returns as soon as one is found. rayon::all checks in parallel whether all elements of the iterator match the predicate, and returns as soon as a non-matching element is found.

extern crate rayon;

use rayon::prelude::*;

fn main() {
    let mut vec = vec![2, 4, 6, 8];

    assert!(!vec.par_iter().any(|n| (*n % 2) != 0)); // None are odd.
    assert!(vec.par_iter().all(|n| (*n % 2) == 0)); // All are even.
    assert!(!vec.par_iter().any(|n| *n > 8 )); // None are greater than 8.
    assert!(vec.par_iter().all(|n| *n <= 8 )); // All are less than or equal to 8.

    vec.push(9);

    assert!(vec.par_iter().any(|n| (*n % 2) != 0)); // At least 1 is odd.
    assert!(!vec.par_iter().all(|n| (*n % 2) == 0)); // Not all are even.
    assert!(vec.par_iter().any(|n| *n > 8 )); // At least 1 is greater than 8.
    assert!(!vec.par_iter().all(|n| *n <= 8 )); // Not all are less than or equal to 8.
}

Sort a vector in parallel

rayon-badge rand-badge cat-concurrency-badge

This example will sort in parallel a vector of Strings.

[1] We start by preallocating a vector of empty Strings, so we can mutate the information in parallel later, to populate the vector with random Strings.

[2] par_iter_mut().for_each takes a closure and applies it in parallel on all the elements of the vector.
[3] Inside the passed closure we modify the element in the vector with a 5 character-long String, random generated.

[4] We have multiple options to sort an Iterable data type, we chose here to use par_sort_unstable because it is usually faster than stable sorting algorithms which rayon also supports.

extern crate rand;
extern crate rayon;

use rand::{Rng, thread_rng};
use rand::distributions::Alphanumeric;
use rayon::prelude::*;

fn main() {
    // [1]
    let mut vec = vec![String::new(); 100_000];

    // [2]
    vec.par_iter_mut().for_each(|p| {
        // [3]
        let mut rng = thread_rng();
        *p = (0..5).map(|_| rng.sample(&Alphanumeric)).collect()
    });

    // [4]
    vec.par_sort_unstable();
}

Map-reduce in parallel

rayon-badge cat-concurrency-badge

This example uses rayon::filter, rayon::map, and rayon::reduce to calculate the conditional average age of a vector of Person objects.

rayon::filter allows us to (in parallel) conditionally include elements from a collection that satisfy the given predicate. Similarly, rayon::map and rayon::reduce allow us to transform the filtered elements via a unary operation and reduce them to a single value via a given binary operation, respectively. We also show use of rayon::sum, which has the same result as the reduce operation in this example.

extern crate rayon;

use rayon::prelude::*;

struct Person {
    age: u32,
}

fn main() {
    let v: Vec<Person> = vec![
        Person { age: 23 },
        Person { age: 19 },
        Person { age: 42 },
        Person { age: 17 },
        Person { age: 17 },
        Person { age: 31 },
        Person { age: 30 },
    ];

    let num_over_30 = v.par_iter().filter(|&x| x.age > 30).count() as f32;
    let sum_over_30 = v.par_iter()
        .map(|x| x.age)
        .filter(|&x| x > 30)
        .reduce(|| 0, |x, y| x + y);

    let alt_sum_30: u32 = v.par_iter()
        .map(|x| x.age)
        .filter(|&x| x > 30)
        .sum();

    let avg_over_30 = sum_over_30 as f32 / num_over_30;
    let alt_avg_over_30 = alt_sum_30 as f32/ num_over_30;

    assert!((avg_over_30 - alt_avg_over_30).abs() < std::f32::EPSILON);
    println!("The average age of people older than 30 is {}", avg_over_30);
}

Search items using given predicate in parallel

rayon-badge cat-concurrency-badge

This example uses rayon::find_any and par_iter to search a vector in parallel for an element satisfying the predicate in the given closure.

If there are multiple elements satisfying the predicate defined in the closure argument of rayon::find_any, we are only guaranteed that one of them will be found, but not necessarily that the first will be found.

Also note that the argument to the closure is a reference to a reference (&&x). Please see the discussion on std::find for additional details.

extern crate rayon;

use rayon::prelude::*;

fn main() {
    let v = vec![6, 2, 1, 9, 3, 8, 11];

    let f1 = v.par_iter().find_any(|&&x| x == 9);
    let f2 = v.par_iter().find_any(|&&x| x % 2 == 0 && x > 6);
    let f3 = v.par_iter().find_any(|&&x| x > 8);

    assert_eq!(f1, Some(&9));
    assert_eq!(f2, Some(&8));
    assert!(f3 > Some(&8));
}

Generate jpg thumbnails in parallel

rayon-badge glob-badge image-badge cat-concurrency-badge cat-filesystem-badge

This example generates thumbnails for all .jpg in the current directory and saves them in a new folder called thumbnails.

Files are found using glob::glob_with to match case insensitively on both .jpg and .JPG. rayon is then used to resize images in parallel using par_iter along with the make_thumbnail() helper function which internally uses DynamicImage::resize.

# #[macro_use]
# extern crate error_chain;
extern crate glob;
extern crate image;
extern crate rayon;

use std::path::Path;
use std::fs::create_dir_all;

# use error_chain::ChainedError;
use glob::{glob_with, MatchOptions};
use image::{FilterType, ImageError};
use rayon::prelude::*;

# error_chain! {
#     foreign_links {
#         Image(ImageError);
#         Io(std::io::Error);
#         Glob(glob::PatternError);
#     }
# }

fn run() -> Result<()> {
    // find all files in current directory that have a .jpg extension
    // use the default MatchOptions so the search is case insensitive
    let options: MatchOptions = Default::default();
    let files: Vec<_> = glob_with("*.jpg", &options)?
        .filter_map(|x| x.ok())
        .collect();

    if files.len() == 0 {
        bail!("No .jpg files found in current directory");
    }

    let thumb_dir = "thumbnails";
    create_dir_all(thumb_dir)?;

    println!("Saving {} thumbnails into '{}'...", files.len(), thumb_dir);

    let image_failures: Vec<_> = files
        .par_iter()
        .map(|path| {
            make_thumbnail(path, thumb_dir, 300)
                .map_err(|e| e.chain_err(|| path.display().to_string()))
        })
        .filter_map(|x| x.err())
        .collect();

    image_failures.iter().for_each(|x| println!("{}", x.display_chain()));

    println!("{} thumbnails saved successfully", files.len() - image_failures.len());
    Ok(())
}

/// Resize `original` to have a maximum dimension of `longest_edge` and save the
/// resized image to the `thumb_dir` folder
fn make_thumbnail<PA, PB>(original: PA, thumb_dir: PB, longest_edge: u32) -> Result<()>
where
    PA: AsRef<Path>,
    PB: AsRef<Path>,
{
    let img = image::open(original.as_ref())?;
    let file_path = thumb_dir.as_ref().join(original);

    Ok(img.resize(longest_edge, longest_edge, FilterType::Nearest)
          .save(file_path)?)
}
#
# quick_main!(run);

Spawn a short-lived thread

crossbeam-badge cat-concurrency-badge

The example uses the crossbeam crate, which provides data structures and functions for concurrent and parallel programming. Scope::spawn spawns a new scoped thread that is guaranteed to terminate before returning from the closure that passed into crossbeam::scope function, meaning that you can reference data from the calling function.

extern crate crossbeam;

use std::cmp;

fn main() {
    let arr = &[-4, 1, 10, 25];
    let max = find_max(arr, 0, arr.len());
    assert_eq!(25, max);
}

fn find_max(arr: &[i32], start: usize, end: usize) -> i32 {
    // Perform sequential computation if there are only a few elements.
    const THRESHOLD: usize = 2;
    if end - start <= THRESHOLD {
        return *arr.iter().max().unwrap();
    }

    let mid = start + (end - start) / 2;
    crossbeam::scope(|scope| {
        let left = scope.spawn(|| find_max(arr, start, mid));
        let right = scope.spawn(|| find_max(arr, mid, end));

        cmp::max(left.join(), right.join())
    })
}

Draw fractal dispatching work to a thread pool

threadpool-badge num-badge num_cpus-badge image-badge cat-concurrency-badgecat-science-badgecat-rendering-badge

This example draws a fractal from Julia set to an image utilizing a thread pool for computation.

Firstly, the example allocates memory for output image of given width and height with ImageBuffer::new and pre-calculates all possible RGB pixel values using Rgb::from_channels. Secondly, creates a new ThreadPool with thread count equal to number of logical cores in CPU obtained with num_cpus::get. Subsequently, dispatches calculation to thread pool ThreadPool::execute.

Lastly, collects calculation results via mpsc::channel with Receiver::recv, draws them with ImageBuffer::put_pixel and encodes the final image into output.png using ImageBuffer::save.

# #[macro_use]
# extern crate error_chain;
extern crate threadpool;
extern crate num;
extern crate num_cpus;
extern crate image;

use std::sync::mpsc::{channel, RecvError};
use threadpool::ThreadPool;
use num::complex::Complex;
use image::{ImageBuffer, Pixel, Rgb};
#
# error_chain! {
#     foreign_links {
#         MpscRecv(RecvError);
#         Io(std::io::Error);
#     }
# }
#
# // Function converting intensity values to RGB
# // Based on http://www.efg2.com/Lab/ScienceAndEngineering/Spectra.htm
# fn wavelength_to_rgb(wavelength: u32) -> Rgb<u8> {
#     let wave = wavelength as f32;
#
#     let (r, g, b) = match wavelength {
#         380...439 => ((440. - wave) / (440. - 380.), 0.0, 1.0),
#         440...489 => (0.0, (wave - 440.) / (490. - 440.), 1.0),
#         490...509 => (0.0, 1.0, (510. - wave) / (510. - 490.)),
#         510...579 => ((wave - 510.) / (580. - 510.), 1.0, 0.0),
#         580...644 => (1.0, (645. - wave) / (645. - 580.), 0.0),
#         645...780 => (1.0, 0.0, 0.0),
#         _ => (0.0, 0.0, 0.0),
#     };
#
#     let factor = match wavelength {
#         380...419 => 0.3 + 0.7 * (wave - 380.) / (420. - 380.),
#         701...780 => 0.3 + 0.7 * (780. - wave) / (780. - 700.),
#         _ => 1.0,
#     };
#
#     let (r, g, b) = (normalize(r, factor), normalize(g, factor), normalize(b, factor));
#     Rgb::from_channels(r, g, b, 0)
# }
#
# // Maps Julia set distance estimation to intensity values
# fn julia(c: Complex<f32>, x: u32, y: u32, width: u32, height: u32, max_iter: u32) -> u32 {
#     let width = width as f32;
#     let height = height as f32;
#
#     let mut z = Complex {
#         // scale and translate the point to image coordinates
#         re: 3.0 * (x as f32 - 0.5 * width) / width,
#         im: 2.0 * (y as f32 - 0.5 * height) / height,
#     };
#
#     let mut i = 0;
#     for t in 0..max_iter {
#         if z.norm() >= 2.0 {
#             break;
#         }
#         z = z * z + c;
#         i = t;
#     }
#     i
# }
#
# // Normalizes color intensity values within RGB range
# fn normalize(color: f32, factor: f32) -> u8 {
#     ((color * factor).powf(0.8) * 255.) as u8
# }

fn run() -> Result<()> {
    let (width, height) = (1920, 1080);
    let mut img = ImageBuffer::new(width, height);
    let iterations = 300;

    // constants to tweak for appearance
    let c = Complex::new(-0.8, 0.156);

    let pool = ThreadPool::new(num_cpus::get());
    let (tx, rx) = channel();

    for y in 0..height {
        let tx = tx.clone();
        pool.execute(move || for x in 0..width {
                         let i = julia(c, x, y, width, height, iterations);
                         let pixel = wavelength_to_rgb(380 + i * 400 / iterations);
                         tx.send((x, y, pixel)).expect("Could not send data!");
                     });
    }

    for _ in 0..(width * height) {
        let (x, y, pixel) = rx.recv()?;
        img.put_pixel(x, y, pixel);
    }
    let _ = img.save("output.png")?;
    Ok(())
}
#
# quick_main!(run);

Calculate SHA1 sum of *.iso files concurrently

threadpool-badge num_cpus-badge walkdir-badge ring-badge cat-concurrency-badgecat-filesystem-badge

This example calculates the SHA1 for every file present in the current directory. A threadpool is created using the number of cpus present in the system with num_cpus::get. Then every returned by Walkdir::new is passed into this pool to perform the operations of reading and computing SHA1. At the end the program waits for all jobs to finish. To get better results, compile this program in release mode.

# #[macro_use]
# extern crate error_chain;
extern crate walkdir;
extern crate ring;
extern crate num_cpus;
extern crate threadpool;

# error_chain! {
#     foreign_links {
#         Io(std::io::Error);
#     }
# }
#
use walkdir::WalkDir;
use std::fs::File;
use std::io::{BufReader, Read};
use std::path::Path;
use threadpool::ThreadPool;
use std::sync::mpsc::channel;
use ring::digest::{Context, Digest, SHA1};

# // Verify the iso extension
# fn is_iso(entry: &Path) -> bool {
#     match entry.extension() {
#         Some(e) if e.to_string_lossy().to_lowercase() == "iso" => true,
#         _ => false,
#     }
# }
#
fn compute_digest<P: AsRef<Path>>(filepath: P) -> Result<(Digest, P)> {
    let mut buf_reader = BufReader::new(File::open(&filepath)?);
    let mut context = Context::new(&SHA1);
    let mut buffer = [0; 1024];

    loop {
        let count = buf_reader.read(&mut buffer)?;
        if count == 0 {
            break;
        }
        context.update(&buffer[..count]);
    }

    Ok((context.finish(), filepath))
}

fn run() -> Result<()> {
    let pool = ThreadPool::new(num_cpus::get());

    let (tx, rx) = channel();

    // Look in the current directory.
    for entry in WalkDir::new("/home/user/Downloads")
        .follow_links(true)
        .into_iter()
        .filter_map(|e| e.ok())
        .filter(|e| !e.path().is_dir() && is_iso(e.path())) {
            let path = entry.path().to_owned();
            let tx = tx.clone();
            pool.execute(move || {
                let digest = compute_digest(path);
                tx.send(digest).expect("Could not send data!");
            });
        }

    drop(tx);
    for t in rx.iter() {
        let (sha, path) = t?;
        println!("{:?} {:?}", sha, path);
    }
    Ok(())
}
#
# quick_main!(run);

Networking

Recipe Crates Categories
Parse a URL from a string to a Url type url-badge cat-net-badge
Create a base URL by removing path segments url-badge cat-net-badge
Create new URLs from a base URL url-badge cat-net-badge
Extract the URL origin (scheme / host / port) url-badge cat-net-badge
Remove fragment identifiers and query pairs from a URL url-badge cat-net-badge
Make a HTTP GET request reqwest-badge cat-net-badge
Download a file to a temporary directory reqwest-badge tempdir-badge cat-net-badge cat-filesystem-badge
Query the GitHub API reqwest-badge serde-badge cat-net-badge cat-encoding-badge
Consume a paginated RESTful API reqwest-badge serde-badge cat-net-badge cat-encoding-badge
Check if an API resource exists reqwest-badge cat-net-badge
Set custom headers and URL parameters for a REST request reqwest-badge hyper-badge url-badge cat-net-badge
Create and delete Gist with GitHub API reqwest-badge serde-badge cat-net-badge cat-encoding-badge
POST a file to paste-rs reqwest-badge cat-net-badge
Listen on unused port TCP/IP std-badge cat-net-badge
Extract all links from a webpage HTML reqwest-badge select-badge cat-net-badge
Check webpage for broken links reqwest-badge select-badge url-badge cat-net-badge
Extract all unique links from a MediaWiki markup reqwest-badge regex-badge cat-net-badge
Make a partial download with HTTP range headers reqwest-badge cat-net-badge
Handle a rate-limited API reqwest-badge hyper-badge cat-net-badge
Parse the MIME type of a HTTP response mime-badge reqwest-badge cat-net-badge cat-encoding-badge

Parse a URL from a string to a Url type

url-badge cat-net-badge

The parse method from the url crate validates and parses a &str into a Url struct. The input string may be malformed so this method returns Result<Url, ParseError>.

Once the URL has been parsed, it can be used with all of the methods on the Url type.

The URL in this code parses successfully, but swapping it out for a malformed URL will print a message containing an explanation of what went wrong.

# #[macro_use]
# extern crate error_chain;
extern crate url;

use url::Url;
#
# error_chain! {
#     foreign_links {
#         UrlParse(url::ParseError);
#     }
# }

fn run() -> Result<()> {
    let s = "https://github.com/rust-lang/rust/issues?labels=E-easy&state=open";

    let parsed = Url::parse(s)?;
    println!("The path part of the URL is: {}", parsed.path());

    Ok(())
}
#
# quick_main!(run);

Create a base URL by removing path segments

url-badge cat-net-badge

# #[macro_use]
# extern crate error_chain;
extern crate url;

use url::Url;
#
# error_chain! {
#     foreign_links {
#         UrlParse(url::ParseError);
#     }
#     errors {
#         CannotBeABase
#     }
# }

fn run() -> Result<()> {
    let full = "https://github.com/rust-lang/cargo?asdf";

    let url = Url::parse(full)?;
    let base = base_url(url)?;

    assert_eq!(base.as_str(), "https://github.com/");
    println!("The base of the URL is: {}", base);

    Ok(())
}

/// Returns the base of the given URL - the part not including any path segments
/// and query parameters.
fn base_url(mut url: Url) -> Result<Url> {
    // Clear path segments.
    match url.path_segments_mut() {
        Ok(mut path) => {
            path.clear();
        }
        Err(_) => {
            // Certain URLs cannot be turned into a base URL.
            return Err(Error::from_kind(ErrorKind::CannotBeABase));
        }
    }

    // Clear query parameters.
    url.set_query(None);

    Ok(url)
}
#
# quick_main!(run);

Create new URLs from a base URL

url-badge cat-net-badge

The join method creates a new URL from a base and relative path.

# #[macro_use]
# extern crate error_chain;
extern crate url;

use url::Url;
#
# error_chain! {
#     foreign_links {
#         UrlParse(url::ParseError);
#     }
# }

fn run() -> Result<()> {
    let path = "/rust-lang/cargo";

    let gh = build_github_url(path)?;

    assert_eq!(gh.as_str(), "https://github.com/rust-lang/cargo");
    println!("The joined URL is: {}", gh);

    Ok(())
}

fn build_github_url(path: &str) -> Result<Url> {
    // Hardcoded in our program. Caller's path will be joined to this.
    const GITHUB: &'static str = "https://github.com";

    let base = Url::parse(GITHUB).expect("hardcoded URL is known to be valid");
    let joined = base.join(path)?;

    Ok(joined)
}
#
# quick_main!(run);

Extract the URL origin (scheme / host / port)

url-badge cat-net-badge

The Url struct exposes various methods to extract information about the URL it represents.

# #[macro_use]
# extern crate error_chain;
extern crate url;

use url::{Url, Host};

# error_chain! {
#     foreign_links {
#         UrlParse(url::ParseError);
#     }
# }
#
fn run() -> Result<()> {
    let s = "ftp://rust-lang.org/examples";

    let url = Url::parse(s)?;

    assert_eq!(url.scheme(), "ftp");
    assert_eq!(url.host(), Some(Host::Domain("rust-lang.org")));
    assert_eq!(url.port_or_known_default(), Some(21));
    println!("The origin is as expected!");

    Ok(())
}
#
# quick_main!(run);

The same result can be obtained using the origin method as well.

# #[macro_use]
# extern crate error_chain;
extern crate url;

use url::{Url, Origin, Host};

# error_chain! {
#     foreign_links {
#         UrlParse(url::ParseError);
#     }
# }
#
fn run() -> Result<()> {
    let s = "ftp://rust-lang.org/examples";

    let url = Url::parse(s)?;

    let expected_scheme = "ftp".to_owned();
    let expected_host = Host::Domain("rust-lang.org".to_owned());
    let expected_port = 21;
    let expected = Origin::Tuple(expected_scheme, expected_host, expected_port);

    let origin = url.origin();
    assert_eq!(origin, expected);
    println!("The origin is as expected!");

    Ok(())
}
#
# quick_main!(run);

Remove fragment identifiers and query pairs from a URL

url-badge cat-net-badge

Parses Url and slices it with url::Position to strip unneeded URL parts.

# #[macro_use]
# extern crate error_chain;
extern crate url;

use url::{Url, Position};
#
# error_chain! {
#     foreign_links {
#         UrlParse(url::ParseError);
#     }
# }

fn run() -> Result<()> {
    let parsed = Url::parse("https://github.com/rust-lang/rust/issues?labels=E-easy&state=open")?;
    let cleaned: &str = &parsed[..Position::AfterPath];
    println!("cleaned: {}", cleaned);
    Ok(())
}
#
# quick_main!(run);

Make a HTTP GET request

reqwest-badge cat-net-badge

Parses the supplied URL and makes a synchronous HTTP GET request with reqwest::get. Prints obtained reqwest::Response status and headers subsequently reading HTTP response body into an allocated String via read_to_string.

# #[macro_use]
# extern crate error_chain;
extern crate reqwest;

use std::io::Read;
#
# error_chain! {
#     foreign_links {
#         Io(std::io::Error);
#         HttpRequest(reqwest::Error);
#     }
# }

fn run() -> Result<()> {
    let mut res = reqwest::get("http://httpbin.org/get")?;
    let mut body = String::new();
    res.read_to_string(&mut body)?;

    println!("Status: {}", res.status());
    println!("Headers:\n{}", res.headers());
    println!("Body:\n{}", body);

    Ok(())
}
#
# quick_main!(run);

Download a file to a temporary directory

reqwest-badge tempdir-badge cat-net-badge cat-filesystem-badge

Creates a temporary directory with TempDir::new and synchronously downloads a file over HTTP using reqwest::get. Creates a target File with name obtained from Response::url within TempDir::path and copies downloaded data into it with io::copy. The temporary directory is automatically removed on run function return.

# #[macro_use]
# extern crate error_chain;
extern crate reqwest;
extern crate tempdir;

use std::io::copy;
use std::fs::File;
use tempdir::TempDir;
#
# error_chain! {
#     foreign_links {
#         Io(std::io::Error);
#         HttpRequest(reqwest::Error);
#     }
# }

fn run() -> Result<()> {
    // create a temp dir with prefix "example"
    let tmp_dir = TempDir::new("example")?;
    // make HTTP request for remote content
    let target = "https://www.rust-lang.org/logos/rust-logo-512x512.png";
    let mut response = reqwest::get(target)?;

    let mut dest = {
        // extract target filename from URL
        let fname = response
            .url()
            .path_segments()
            .and_then(|segments| segments.last())
            .and_then(|name| if name.is_empty() { None } else { Some(name) })
            .unwrap_or("tmp.bin");

        println!("file to download: '{}'", fname);
        let fname = tmp_dir.path().join(fname);
        println!("will be located under: '{:?}'", fname);
        // create file with given name inside the temp dir
        File::create(fname)?
    };
    // data is copied into the target file
    copy(&mut response, &mut dest)?;
    // tmp_dir is implicitly removed
    Ok(())
}
#
# quick_main!(run);

Query the GitHub API

reqwest-badge serde-badge cat-net-badge cat-encoding-badge

Queries GitHub stargazers API v3 with reqwest::get to get list of all users who have marked a GitHub project with a star. reqwest::Response is deserialized with Response::json into User objects implementing serde::Deserialize.

# #[macro_use]
# extern crate error_chain;
#[macro_use]
extern crate serde_derive;
extern crate reqwest;

#[derive(Deserialize, Debug)]
struct User {
    login: String,
    id: u32,
    // remaining fields not deserialized for brevity
}
#
# error_chain! {
#     foreign_links {
#         Reqwest(reqwest::Error);
#     }
# }

fn run() -> Result<()> {
    let request_url = format!("https://api.github.com/repos/{owner}/{repo}/stargazers",
                              owner = "rust-lang-nursery",
                              repo = "rust-cookbook");
    println!("{}", request_url);
    let mut response = reqwest::get(&request_url)?;

    let users: Vec<User> = response.json()?;
    println!("{:?}", users);
    Ok(())
}
#
# quick_main!(run);

Check if an API resource exists

reqwest-badge cat-net-badge

Query the GitHub Users Endpoint using a HEAD request (Client::head) and then inspect the response code to determine success. This is a quick way to query a rest resource without needing to receive a body. You can also configure the reqwest::Client with ClientBuilder::timeout which ensures that a request will not last longer than what is passed to the timeout function.

# #[macro_use]
# extern crate error_chain;
extern crate reqwest;

use std::time::Duration;
use reqwest::ClientBuilder;
#
# error_chain! {
#     foreign_links {
#         Reqwest(reqwest::Error);
#     }
# }

fn run() -> Result<()> {
    let user = "ferris-the-crab";
    let request_url = format!("https://api.github.com/users/{}", user);
    println!("{}", request_url);

    // The timeout for the request is set to 5 seconds.
    let timeout = Duration::new(5, 0);

    let client = ClientBuilder::new().timeout(timeout).build()?;
    let response = client.head(&request_url).send()?;

    if response.status().is_success() {
        println!("{} is a user!", user);
    } else {
        println!("{} is not a user!", user);
    }

    Ok(())
}
#
# quick_main!(run);

Set custom headers and URL parameters for a REST request

reqwest-badge hyper-badge url-badge cat-net-badge

Sets both standard and custom HTTP headers as well as URL parameters for HTTP GET request. Firstly creates a custom header of type XPoweredBy with hyper::header! macro. Secondly calls Url::parse_with_params in order to build a complex URL with specified key value pairs. Lastly sets standard headers header::UserAgent and header::Authorization as well as custom one XPoweredBy with RequestBuilder::header prior to making the request with RequestBuilder::send.

The code is run against http://httpbin.org/headers service which responds with a JSON dict containing all request headers for easy verification.

# #[macro_use]
# extern crate error_chain;
extern crate url;
extern crate reqwest;
#[macro_use]
extern crate hyper;
#[macro_use]
extern crate serde_derive;

use std::collections::HashMap;
use url::Url;
use reqwest::Client;
use reqwest::header::{UserAgent, Authorization, Bearer};

// Custom header type
header! { (XPoweredBy, "X-Powered-By") => [String] }

// Helper for verification
#[derive(Deserialize, Debug)]
pub struct HeadersEcho {
    pub headers: HashMap<String, String>,
}
#
# error_chain! {
#     foreign_links {
#         Reqwest(reqwest::Error);
#         UrlParse(url::ParseError);
#     }
# }

fn run() -> Result<()> {
    // Make request to webservice that will respond with JSON dict containing
    // the headders set on HTTP GET request.
    let url = Url::parse_with_params("http://httpbin.org/headers",
                                     &[("lang", "rust"), ("browser", "servo")])?;

    let mut response = Client::new()
        .get(url)
        .header(UserAgent::new("Rust-test"))
        .header(Authorization(Bearer { token: "DEadBEEfc001cAFeEDEcafBAd".to_owned() }))
        .header(XPoweredBy("Guybrush Threepwood".to_owned()))
        .send()?;

    // JSON response should match the headers set on request
    let out: HeadersEcho = response.json()?;
    assert_eq!(out.headers["Authorization"],
               "Bearer DEadBEEfc001cAFeEDEcafBAd");
    assert_eq!(out.headers["User-Agent"], "Rust-test");
    assert_eq!(out.headers["X-Powered-By"], "Guybrush Threepwood");
    // Response contains full URL used to make the request
    assert_eq!(response.url().as_str(),
               "http://httpbin.org/headers?lang=rust&browser=servo");

    println!("{:?}", out);
    Ok(())
}
#
# quick_main!(run);

Create and delete Gist with GitHub API

reqwest-badge serde-badge cat-net-badge cat-encoding-badge

Creates a gist with POST request to GitHub gists API v3 using Client::post and subsequently removes it with DELETE request using Client::delete.

The reqwest::Client is responsible for details of both requests including URL, body and authentication. POST body comes from serde_json::json! macro which provides a way to pass an arbitrary JSON body. Call to RequestBuilder::json sets the request body while RequestBuilder::basic_auth handles authentication. Finally the call to RequestBuilder::send synchronously executes the requests.

# #[macro_use]
# extern crate error_chain;
extern crate reqwest;
#[macro_use]
extern crate serde_derive;
#[macro_use]
extern crate serde_json;

use std::env;
use reqwest::Client;
#
# error_chain! {
#     foreign_links {
#         EnvVar(env::VarError);
#         HttpRequest(reqwest::Error);
#     }
# }

#[derive(Deserialize, Debug)]
struct Gist {
    id: String,
    html_url: String,
    // remaining fields not deserialized for brevity
}

fn run() -> Result<()> {
    let gh_user = env::var("GH_USER")?;
    let gh_pass = env::var("GH_PASS")?;

    // The type `gist_body` is `serde_json::Value`
    let gist_body = json!({
        "description": "the description for this gist",
        "public": true,
        "files": {
             "main.rs": {
             "content": r#"fn main() { println!("hello world!");}"#
            }
        }});

    // create the gist
    let request_url = "https://api.github.com/gists";
    let mut response = Client::new()
        .post(request_url)
        .basic_auth(gh_user.clone(), Some(gh_pass.clone()))
        .json(&gist_body)
        .send()?;

    let gist: Gist = response.json()?;
    println!("Created {:?}", gist);

    // delete the gist
    let request_url = format!("{}/{}",request_url, gist.id);
    let response = Client::new()
        .delete(&request_url)
        .basic_auth(gh_user, Some(gh_pass))
        .send()?;

    println!("Gist {} deleted! Status code: {}",gist.id, response.status());
    Ok(())
}
#
# quick_main!(run);

For the sake of simplicity the example uses HTTP Basic Auth in order to authorize access to GitHub API. A more typical use case would be to employ one of the much more complex OAuth authorization flows.

Consume a paginated RESTful API

reqwest-badge serde-badge cat-net-badge cat-encoding-badge

Wraps a paginated web API in a convenient Rust iterator. The iterator lazily fetches the next page of results from the remote server as it arrives at the end of each page.

# #[macro_use]
# extern crate error_chain;
#[macro_use]
extern crate serde_derive;
extern crate reqwest;
#
# error_chain! {
#     foreign_links {
#         Reqwest(reqwest::Error);
#     }
# }

#[derive(Deserialize)]
struct ApiResponse {
    dependencies: Vec<Dependency>,
    meta: Meta,
}

// Could capture more fields here if needed
#[derive(Deserialize)]
struct Dependency {
    crate_id: String,
}

#[derive(Deserialize)]
struct Meta {
    total: u32,
}

struct ReverseDependencies {
    crate_id: String,
    dependencies: <Vec<Dependency> as IntoIterator>::IntoIter,
    client: reqwest::Client,
    page: u32,
    per_page: u32,
    total: u32,
}

impl ReverseDependencies {
    fn of(crate_id: &str) -> Result<Self> {
        Ok(ReverseDependencies {
               crate_id: crate_id.to_owned(),
               dependencies: vec![].into_iter(),
               client: reqwest::Client::new(),
               page: 0,
               per_page: 100,
               total: 0,
           })
    }

    fn try_next(&mut self) -> Result<Option<Dependency>> {
        // If the previous page has a dependency that hasn't been looked at.
        if let Some(dep) = self.dependencies.next() {
            return Ok(Some(dep));
        }

        // If there are no more reverse dependencies.
        if self.page > 0 && self.page * self.per_page >= self.total {
            return Ok(None);
        }

        // Fetch the next page.
        self.page += 1;
        let url = format!("https://crates.io/api/v1/crates/{}/reverse_dependencies?page={}&per_page={}",
                          self.crate_id,
                          self.page,
                          self.per_page);

        let response = self.client.get(&url).send()?.json::<ApiResponse>()?;
        self.dependencies = response.dependencies.into_iter();
        self.total = response.meta.total;
        Ok(self.dependencies.next())
    }
}

impl Iterator for ReverseDependencies {
    type Item = Result<Dependency>;

    fn next(&mut self) -> Option<Self::Item> {
        // Some juggling required here because `try_next` returns a result
        // containing an option, while `next` is supposed to return an option
        // containing a result.
        match self.try_next() {
            Ok(Some(dep)) => Some(Ok(dep)),
            Ok(None) => None,
            Err(err) => Some(Err(err)),
        }
    }
}

fn run() -> Result<()> {
    for dep in ReverseDependencies::of("serde")? {
        println!("reverse dependency: {}", dep?.crate_id);
    }
    Ok(())
}
#
# quick_main!(run);

POST a file to paste-rs.

reqwest-badge cat-net-badge

A connection is established to https://paste.rs using reqwest::Client, following the reqwest::RequestBuilder pattern. Calling Client::post with a URL establishes the destination, RequestBuilder::body sets the content to send by reading the file, and RequestBuilder::send blocks until the file uploads and the response is received. The response is read with read_to_string, and finally displayed in the console.

extern crate reqwest;

# #[macro_use]
# extern crate error_chain;
#
use std::fs::File;
use std::io::Read;
use reqwest::Client;
#
# error_chain! {
#     foreign_links {
#         HttpRequest(reqwest::Error);
#         IoError(::std::io::Error);
#     }
# }

fn run() -> Result<()> {
    let paste_api = "https://paste.rs";
    let file = File::open("message")?;

    // blocks until paste.rs returns a response
    let mut response = Client::new().post(paste_api).body(file).send()?;
    let mut response_body = String::new();
    response.read_to_string(&mut response_body)?;
    println!("Your paste is located at: {}", response_body);
    Ok(())
}
#
# quick_main!(run);

Listen on unused port TCP/IP

std-badge cat-net-badge

In this example, the port is displayed on the console, and the program will listen until a request is made.

# #[macro_use]
# extern crate error_chain;
#
use std::net::{SocketAddrV4, Ipv4Addr, TcpListener};
use std::io::Read;
#
# error_chain! {
#     foreign_links {
#         Io(::std::io::Error);
#     }
# }

fn run() -> Result<()> {
    let loopback = Ipv4Addr::new(127, 0, 0, 1);
    // Assigning port 0 requests the OS to assign a free port
    let socket = SocketAddrV4::new(loopback, 0);
    let listener = TcpListener::bind(socket)?;
    let port = listener.local_addr()?;
    println!("Listening on {}, access this port to end the program", port);
    let (mut tcp_stream, addr) = listener.accept()?; //block  until requested
    println!("Connection received! {:?} is sending data.", addr);
    let mut input = String::new();
    // read from the socket until connection closed by client, discard byte count.
    let _ = tcp_stream.read_to_string(&mut input)?;
    println!("{:?} says {}", addr, input);
    Ok(())
}
#
# quick_main!(run);

The std library is leveraged to make a well formed IP/port with the SocketAddrV4 and Ipv4Addr structs. An unused random port is requested by passing 0 to TcpListener::bind. The assigned address is available via TcpListener::local_addr.

TcpListener::accept synchronously waits for an incoming connection and returns a (TcpStream, SocketAddrV4) representing the request. Reading on the socket with read_to_string will wait until the connection is closed which can be tested with telnet ip port. For example, if the program shows Listening on 127.0.0.1:11500, run

telnet 127.0.0.1 11500

After sending data in telnet press ctrl-] and type quit.

Extract all links from a webpage HTML

reqwest-badge select-badge cat-net-badge

Use reqwest::get to perform a HTTP GET request and then use Document::from_read to parse the response into a HTML document. We can then retrieve all the links from the document by using find with the criteria of the Name being "a". This returns a Selection that we filter_map on to retrieve the urls from links that have the "href" attr.

# #[macro_use]
# extern crate error_chain;
extern crate reqwest;
extern crate select;

use select::document::Document;
use select::predicate::Name;
#
# error_chain! {
#    foreign_links {
#        ReqError(reqwest::Error);
#        IoError(std::io::Error);
#    }
# }

fn run() -> Result<()> {
    let res = reqwest::get("https://www.rust-lang.org/en-US/")?;

    Document::from_read(res)?
        .find(Name("a"))
        .filter_map(|n| n.attr("href"))
        .for_each(|x| println!("{}", x));

    Ok(())
}
#
# quick_main!(run);

Check a webpage for broken links

reqwest-badge select-badge url-badge cat-net-badge

We call "get_base_url" to retrieve the base URL. If the document has a "base" tag, we get the "href" attr from the first occurrence of the "base" tag. This is then used as the base URL. Otherwise, we can use Position::BeforePath with the original URL to get the base of that URL.

We iterate through all the links in the document and get the absolute URL for each (using url::ParseOptions and Url::parse). We then filter these so that we can report which links are broken.

# #[macro_use]
# extern crate error_chain;
extern crate reqwest;
extern crate select;
extern crate url;

use std::collections::HashSet;

use url::{Url, Position};
use reqwest::StatusCode;
use select::document::Document;
use select::predicate::Name;
#
# error_chain! {
#   foreign_links {
#       ReqError(reqwest::Error);
#       IoError(std::io::Error);
#       UrlParseError(url::ParseError);
#   }
# }

fn get_base_url(url: &Url, doc: &Document) -> Result<Url> {
    let base_tag_href = doc.find(Name("base")).filter_map(|n| n.attr("href")).nth(0);

    let base_url = base_tag_href.map_or_else(
        || Url::parse(&url[..Position::BeforePath]),
        Url::parse,
    )?;

    Ok(base_url)
}

fn check_link(url: &Url) -> Result<bool> {
    let res = reqwest::get(url.as_ref())?;

    Ok(res.status() != StatusCode::NotFound)
}

fn run() -> Result<()> {
    let url = Url::parse("https://www.rust-lang.org/en-US/")?;

    let res = reqwest::get(url.as_ref())?;
    let document = Document::from_read(res)?;

    let base_url = get_base_url(&url, &document)?;

    let base_parser = Url::options().base_url(Some(&base_url));

    let links: HashSet<Url> = document
        .find(Name("a"))
        .filter_map(|n| n.attr("href"))
        .filter_map(|link| base_parser.parse(link).ok())
        .collect();

    links
        .iter()
        .filter(|link| check_link(link).ok() == Some(false))
        .for_each(|x| println!("{} is broken.", x));

    Ok(())
}
#
# quick_main!(run);

Extract all unique links from a MediaWiki markup

reqwest-badge regex-badge cat-net-badge

Pull the source of a MediaWiki page using reqwest::get and then look for all entries of internal and external links with Regex::captures_iter. Using Cow avoids excessive String allocations.

MediaWiki link syntax is described here.

# #[macro_use]
# extern crate error_chain;
#[macro_use]
extern crate lazy_static;
extern crate reqwest;
extern crate regex;

use std::io::Read;
use std::collections::HashSet;
use std::borrow::Cow;
use regex::Regex;

# error_chain! {
#     foreign_links {
#         Io(std::io::Error);
#         Reqwest(reqwest::Error);
#         Regex(regex::Error);
#     }
# }
#
fn extract_links(content: &str) -> Result<HashSet<Cow<str>>> {
    lazy_static! {
        static ref WIKI_REGEX: Regex =
            Regex::new(r"(?x)
                \[\[(?P<internal>[^\[\]|]*)[^\[\]]*\]\]    # internal links
                |
                (url=|URL\||\[)(?P<external>http.*?)[ \|}] # external links
            ").unwrap();
    }

    let links: HashSet<_> = WIKI_REGEX
        .captures_iter(content)
        .map(|c| match (c.name("internal"), c.name("external")) {
            (Some(val), None) => Cow::from(val.as_str().to_lowercase()),
            (None, Some(val)) => Cow::from(val.as_str()),
            _ => unreachable!(),
        })
        .collect();

    Ok(links)
}

fn run() -> Result<()> {
    let mut content = String::new();
    reqwest::get(
        "https://en.wikipedia.org/w/index.php?title=Rust_(programming_language)&action=raw",
    )?
        .read_to_string(&mut content)?;

    println!("{:#?}", extract_links(&content)?);

    Ok(())
}
#
# quick_main!(run);

Make a partial download with HTTP range headers

reqwest-badge cat-net-badge

Uses reqwest::Client::head to get the content-length and validate if the server sets the header reqwest::header::ContentRange, required to confirm the support of partial downloads.

If supported, downloads the content using reqwest::get, setting the reqwest::header::Range to do partial downloads printing basic progress messages. in chunks of 10240 bytes

Range header is defined in RFC7233.

# #[macro_use]
# extern crate error_chain;
extern crate reqwest;

use std::fs::File;
use reqwest::header::{ContentRange, ContentRangeSpec, Range};
use reqwest::StatusCode;
#
# error_chain! {
#     foreign_links {
#         Io(std::io::Error);
#         Reqwest(reqwest::Error);
#     }
# }
#
# struct PartialRangeIter {
#     start: u64,
#     end: u64,
#     buffer_size: u32,
# }
#
# impl PartialRangeIter {
#     pub fn new(content_range: &ContentRangeSpec, buffer_size: u32) -> Result<Self> {
#         if buffer_size == 0 {
#             Err("invalid buffer_size, give a value greater than zero.")?;
#         }
#
#         match *content_range {
#             ContentRangeSpec::Bytes { range: Some(range), .. } => Ok(PartialRangeIter {
#                 start: range.0,
#                 end: range.1,
#                 buffer_size,
#             }),
#             _ => Err("invalid range specification")?,
#         }
#     }
# }
#
# impl Iterator for PartialRangeIter {
#     type Item = Range;
#
#     fn next(&mut self) -> Option<Self::Item> {
#         if self.start > self.end {
#             None
#         } else {
#             let prev_start = self.start;
#             self.start += std::cmp::min(self.buffer_size as u64, self.end - self.start + 1);
#             Some(Range::bytes(prev_start, self.start - 1))
#         }
#     }
# }

fn run() -> Result<()> {
    // For the purpose of this example only a small download of 102400 bytes
    // with chunk size of 10240 bytes is used.
    let url = "https://httpbin.org/range/102400?duration=2";
    const CHUNK_SIZE: u32 = 10240;

    let client = reqwest::Client::new();
    let response = client.head(url).send()?;
    let range = response.headers().get::<ContentRange>().ok_or(
        "response doesn't include the expected ranges",
    )?;

    let mut output_file = File::create("download.bin")?;

    println!("starting download...");
    for range in PartialRangeIter::new(range, CHUNK_SIZE)? {

        println!("range {:?}", range);
        let mut response = client.get(url).header(range).send()?;

        let status = response.status();
        if !(status == StatusCode::Ok || status == StatusCode::PartialContent) {
            bail!("Unexpected server response: {}", status)
        }

        std::io::copy(&mut response, &mut output_file)?;
    }

    println!("Finished with success!");
    Ok(())
}
#
# quick_main!(run);

Handle a rate-limited API

reqwest-badge hyper-badge cat-net-badge

This example uses the GitHub API - Rate limiting, as an example of how to handle remote server errors. This example uses the hyper::header! macro to parse the response header and checks for reqwest::StatusCode::Forbidden. If the response exceeds the rate limit, the example waits and retries.

# #[macro_use]
# extern crate error_chain;
#[macro_use]
extern crate hyper;
extern crate reqwest;

use std::time::{Duration, UNIX_EPOCH};
use std::thread;
use reqwest::StatusCode;
#
# error_chain! {
#    foreign_links {
#        Io(std::io::Error);
#        Time(std::time::SystemTimeError);
#        Reqwest(reqwest::Error);
#    }
# }

header! { (XRateLimitLimit, "X-RateLimit-Limit") => [usize] }
header! { (XRateLimitRemaining, "X-RateLimit-Remaining") => [usize] }
header! { (XRateLimitReset, "X-RateLimit-Reset") => [u64] }

fn run() -> Result<()> {
    let url = "https://api.github.com/users/rust-lang-nursery ";
    let client = reqwest::Client::new();
    let response = client.get(url).send()?;

    let rate_limit = response.headers().get::<XRateLimitLimit>().ok_or(
        "response doesn't include the expected X-RateLimit-Limit header",
    )?;

    let rate_remaining = response.headers().get::<XRateLimitRemaining>().ok_or(
        "response doesn't include the expected X-RateLimit-Remaining header",
    )?;

    let rate_reset_at = response.headers().get::<XRateLimitReset>().ok_or(
        "response doesn't include the expected X-RateLimit-Reset header",
    )?;

    let rate_reset_within = Duration::from_secs(**rate_reset_at) - UNIX_EPOCH.elapsed()?;

    if response.status() == StatusCode::Forbidden && **rate_remaining == 0 {
        println!("Sleeping for {} seconds.", rate_reset_within.as_secs());
        thread::sleep(rate_reset_within);
        return run();
    }
    else {
        println!(
            "Rate limit is currently {}/{}, the reset of this limit will be within {} seconds.",
            **rate_remaining,
            **rate_limit,
            rate_reset_within.as_secs(),
        );
    }

    Ok(())
}
#
# quick_main!(run);

Parse the MIME type of a HTTP response

reqwest-badge mime-badge cat-net-badge cat-encoding-badge

When receiving a HTTP reponse from reqwest the MIME type or media type can be found in the Content-Type header. The header can be looked up by using reqwest::Headers::get with the generic type reqwest::header::ContentType. Because ContentType implements Deref with mime::Mime as a target, parts of the MIME type can be obtained directly.

The Mime crate also has some, commonly used, predefined MIME types. These can be used for comparison and matching on the types. Reqwest also exports the mime crate, which can be found in the reqwest::mime module.

# #[macro_use]
# extern crate error_chain;
extern crate mime;
extern crate reqwest;

use reqwest::header::ContentType;
#
# error_chain! {
#    foreign_links {
#        Reqwest(reqwest::Error);
#    }
# }

fn run() -> Result<()> {
    let response = reqwest::get("https://www.rust-lang.org/logos/rust-logo-32x32.png")?;
    let headers = response.headers();

    match headers.get::<ContentType>() {
        None => {
            println!("The response does not contain a Content-Type header.");
        }
        Some(content_type) => {
            let media_type = match (content_type.type_(), content_type.subtype()) {
                (mime::TEXT, mime::HTML) => "a HTML document",
                (mime::TEXT, _) => "a text document",
                (mime::IMAGE, mime::PNG) => "a PNG image",
                (mime::IMAGE, _) => "an image",
                _ => "neither text nor image",
            };

            println!("The reponse contains {}.", media_type);
        }
    };

    Ok(())
}
#
# quick_main!(run);

Application development

Recipe Crates Categories
Parse command line arguments clap-badge cat-command-line-badge
Decompress a tarball flate2-badge tar-badge cat-compression-badge
Compress a directory into a tarball flate2-badge tar-badge cat-compression-badge
Decompress a tarball while removing a prefix from the paths flate2-badge tar-badge cat-compression-badge
Avoid writing and reading from a same file same_file-badge cat-filesystem-badge
Find loops for a given path same_file-badge cat-filesystem-badge
Recursively find duplicate file names walkdir-badge cat-filesystem-badge
Recursively find all files with given predicate walkdir-badge cat-filesystem-badge
Traverse directories while skipping dotfiles walkdir-badge cat-filesystem-badge
Recursively calculate file sizes at given depth walkdir-badge cat-filesystem-badge
Find all png files recursively glob-badge cat-filesystem-badge
Find all files with given pattern ignoring filename case glob-badge cat-filesystem-badge
Parse and increment a version string semver-badge cat-config-badge
Parse a complex version string semver-badge cat-config-badge
Check if given version is pre-release semver-badge cat-config-badge
Find the latest version satisfying given range semver-badge cat-config-badge
Check external command version for compatibility semver-badge cat-text-processing-badge cat-os-badge

Parse command line arguments

clap-badge cat-command-line-badge

extern crate clap;

use clap::{Arg, App};

fn main() {
    // Define command line arguments.
    let matches = App::new("My Test Program")
        .version("0.1.0")
        .author("Hackerman Jones <hckrmnjones@hack.gov>")
        .about("Teaches argument parsing")
        .arg(Arg::with_name("file")
                 .short("f")
                 .long("file")
                 .takes_value(true)
                 .help("A cool file"))
        .arg(Arg::with_name("num")
                 .short("n")
                 .long("number")
                 .takes_value(true)
                 .help("Five less than your favorite number"))
        .get_matches();

    // Get value for file, or default to 'input.txt'.
    let myfile = matches.value_of("file").unwrap_or("input.txt");
    println!("The file passed is: {}", myfile);

    // Get value for num if present, and try parsing it as i32.
    let num_str = matches.value_of("num");
    match num_str {
        None => println!("No idea what your favorite number is."),
        Some(s) => {
            match s.parse::<i32>() {
                Ok(n) => println!("Your favorite number must be {}.", n + 5),
                Err(_) => println!("That's not a number! {}", s),
            }
        }
    }
}

The clap crate is a simple-to-use, efficient, and full-featured library for parsing command line arguments and subcommands when writing console/terminal applications.

The application can describe the structure of its command-line interface using clap's builder style. The documentation gives two other possible ways to instantiate an application.

In the builder style, with_name is the unique identifier that value_of will use to retrieve the value passed. The short and long options control the flag the user will be expected to type; short flags look like -f and long flags look like --file.

Usage information is generated by clap. The usage for the example application looks like this.

My Test Program 0.1.0
Hackerman Jones <hckrmnjones@hack.gov>
Teaches argument parsing

USAGE:
    testing [OPTIONS]

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

OPTIONS:
    -f, --file <file>     A cool file
    -n, --number <num>    Five less than your favorite number

We can test the application by running a command like the following.

$ cargo run -- -f myfile.txt -n 251

The output is:

The file passed is: myfile.txt
Your favorite number must be 256.

Decompress a tarball

flate2-badge tar-badge cat-compression-badge

Decompress (GzDecoder) and extract (Archive::unpack) all files from a compressed tarball named archive.tar.gz located in the current working directory.

# #[macro_use]
# extern crate error_chain;
extern crate flate2;
extern crate tar;

use std::fs::File;
use flate2::read::GzDecoder;
use tar::Archive;
#
# error_chain! {
#     foreign_links {
#         Io(std::io::Error);
#     }
# }

fn run() -> Result<()> {
    let path = "archive.tar.gz";

    // Open a compressed tarball
    let tar_gz = File::open(path)?;
    // Decompress it
    let tar = GzDecoder::new(tar_gz);
    // Load the archive from the tarball
    let mut archive = Archive::new(tar);
    // Unpack the archive inside current working directory
    archive.unpack(".")?;

    Ok(())
}
#
# quick_main!(run);

Compress a directory into tarball

flate2-badge tar-badge cat-compression-badge

Compresses /var/log directory into archive.tar.gz.

Creates a File wrapped in GzEncoder and tar::Builder.
Adds contents of /var/log directory recursively into the archive under backup/logspath with Builder::append_dir_all. GzEncoder is responsible for transparently compressing the data prior to writing it into archive.tar.gz.

# #[macro_use]
# extern crate error_chain;
extern crate tar;
extern crate flate2;
#
# error_chain! {
#     foreign_links {
#         Io(std::io::Error);
#     }
# }

use std::fs::File;
use flate2::Compression;
use flate2::write::GzEncoder;

fn run() -> Result<()> {
    let tar_gz = File::create("archive.tar.gz")?;
    let enc = GzEncoder::new(tar_gz, Compression::default());
    let mut tar = tar::Builder::new(enc);
    tar.append_dir_all("backup/logs", "/var/log")?;
    Ok(())
}
#
# quick_main!(run);

Decompress a tarball while removing a prefix from the paths

flate2-badge tar-badge cat-compression-badge

Strip a path prefix from the entries of a tarball before unpacking them.

We iterate over the Archive::entries, using Path::strip_prefix to remove the specified path prefix (bundle/logs) before extracting the tar::Entry via Entry::unpack.

# #[macro_use]
# extern crate error_chain;
extern crate flate2;
extern crate tar;
#
# error_chain! {
#     foreign_links {
#         Io(std::io::Error);
#         StripPrefixError(::std::path::StripPrefixError);
#     }
# }

use std::fs::File;
use std::path::PathBuf;
use flate2::read::GzDecoder;
use tar::Archive;

fn run() -> Result<()> {
    let file = File::open("archive.tar.gz")?;
    let mut archive = Archive::new(GzDecoder::new(file));
    let prefix = "bundle/logs";

    println!("Extracted the following files:");
    archive
        .entries()?
        .filter_map(|e| e.ok())
        .map(|mut entry| -> Result<PathBuf> {
            // Need to get owned data to break the borrow loop
            let path = entry.path()?.strip_prefix(prefix)?.to_owned();
            entry.unpack(&path)?;
            Ok(path)
        })
        .filter_map(|e| e.ok())
        .for_each(|x| println!("> {}", x.display()));

    Ok(())
}
#
# quick_main!(run);

Avoid writing and reading from a same file

same_file-badge cat-filesystem-badge

Use same_file::Handle to a file that can be tested for equality with other handles. In this example, the handles of file to be read from and to be written to are tested for equality.

cargo run

will display the contents of the file if the two files are not same and

cargo run >> ./new.txt

will display the error (because the two files are same).

# #[macro_use]
# extern crate error_chain;
extern crate same_file;

use same_file::Handle;
use std::path::Path;
use std::fs::File;
use std::io::{BufRead, BufReader};
#
# error_chain! {
#     foreign_links {
#          IOError(::std::io::Error);
#     }
# }

fn run() -> Result<()> {
    let path_to_read = Path::new("new.txt");

    let stdout_handle = Handle::stdout()?;
    let handle = Handle::from_path(path_to_read)?;

    if stdout_handle == handle {
        bail!("You are reading and writing to the same file");
    } else {
        let file = File::open(&path_to_read)?;
        let file = BufReader::new(file);
        for (num, line) in file.lines().enumerate() {
            println!("{} : {}", num, line?.to_uppercase());
        }
    }

    Ok(())
}
#
# quick_main!(run);

Find loops for a given path

same_file-badge cat-filesystem-badge

Use same_file::is_same_file to detect loops for a given path. For example, a loop could be created on a Unix system via symlinks:

mkdir -p /tmp/foo/bar/baz
ln -s /tmp/foo/  /tmp/foo/bar/baz/qux

Then, running the following would assert that there exists a loop.

extern crate same_file;

use std::io;
use std::path::{Path, PathBuf};
use same_file::is_same_file;

// Check this path against all of its parents
fn contains_loop<P: AsRef<Path>>(path: P) -> io::Result<Option<(PathBuf, PathBuf)>> {
    let path = path.as_ref();
    let mut path_buf = path.to_path_buf();
    while path_buf.pop() {
        if is_same_file(&path_buf, path)? {
            return Ok(Some((path_buf, path.to_path_buf())));
        } else if let Some(looped_paths) = contains_loop(&path_buf)? {
            return Ok(Some(looped_paths));
        }
    }
    return Ok(None);
}

fn main() {
    assert_eq!(
        contains_loop("/tmp/foo/bar/baz/qux/bar/baz").unwrap(),
        Some((
            PathBuf::from("/tmp/foo"),
            PathBuf::from("/tmp/foo/bar/baz/qux")
        ))
    );
}

Recursively find duplicate file names

walkdir-badge cat-filesystem-badge

Find recursively in the current directory duplicate filenames, printing them only once.

extern crate walkdir;

use std::collections::HashMap;
use walkdir::WalkDir;

fn main() {
    // Counters indexed by filenames
    let mut filenames = HashMap::new();

    // List recusively all files in the current directory filtering out
    // directories and files not accessible (permission denied)
    for entry in WalkDir::new(".")
            .into_iter()
            .filter_map(Result::ok)
            .filter(|e| !e.file_type().is_dir()) {
        // Get entry's filename
        let f_name = String::from(entry.file_name().to_string_lossy());
        // Get or initialize the counter
        let counter = filenames.entry(f_name.clone()).or_insert(0);
        // Update the counter
        *counter += 1;

        if *counter == 2 {
            println!("{}", f_name);
        }
    }
}

Recursively find all files with given predicate

walkdir-badge cat-filesystem-badge

Find JSON files modified within the last day in the current directory. Using follow_links ensures symbolic links are followed like they were normal directories and files.

# #[macro_use]
# extern crate error_chain;
extern crate walkdir;

use walkdir::WalkDir;
#
# error_chain! {
#     foreign_links {
#         WalkDir(walkdir::Error);
#         Io(std::io::Error);
#         SystemTime(std::time::SystemTimeError);
#     }
# }

fn run() -> Result<()> {
    // List recusively all accessible files in the current directory
    for entry in WalkDir::new(".")
            .follow_links(true)
            .into_iter()
            .filter_map(|e| e.ok()) {
        // Get entry's filename
        let f_name = entry.file_name().to_string_lossy();
        // Get entry's modified time
        let sec = entry.metadata()?.modified()?;

        // Print JSON files modified within the last day
        if f_name.ends_with(".json") && sec.elapsed()?.as_secs() < 86400 {
            println!("{}", f_name);
        }
    }

    Ok(())
}
#
# quick_main!(run);

Traverse directories while skipping dotfiles

walkdir-badge cat-filesystem-badge

Uses filter_entry to descend recursively into entries passing the is_not_hidden predicate thus skipping hidden files and directories whereas Iterator::filter would be applied to each WalkDir::DirEntry even if the parent is a hidden directory.

Root dir "." is yielded due to WalkDir::depth usage in is_not_hidden predicate.

extern crate walkdir;

use walkdir::{DirEntry, WalkDir};

fn is_not_hidden(entry: &DirEntry) -> bool {
    entry
         .file_name()
         .to_str()
         .map(|s| entry.depth() == 0 || !s.starts_with("."))
         .unwrap_or(false)
}

fn main() {
    WalkDir::new(".")
        .into_iter()
        .filter_entry(|e| is_not_hidden(e))
        .filter_map(|v| v.ok())
        .for_each(|x| println!("{}", x.path().display()));
}

Recursively calculate file sizes at given depth

walkdir-badge cat-filesystem-badge

Recursion depth can be flexibly set by WalkDir::min_depth & WalkDir::max_depth methods. In this example we sum all file sizes to 3 subfolders depth, ignoring files in the root folder at the same time.

extern crate walkdir;

use walkdir::WalkDir;

fn main() {
    let total_size = WalkDir::new(".")
        .min_depth(1)
        .max_depth(3)
        .into_iter()
        .filter_map(|entry| entry.ok()) // Files, we have access to
        .filter_map(|entry| entry.metadata().ok()) // Get metadata
        .filter(|metadata| metadata.is_file()) // Filter out directories
        .fold(0, |acc, m| acc + m.len()); // Accumulate sizes

    println!("Total size: {} bytes.", total_size);
}

Find all png files recursively

glob-badge cat-filesystem-badge

Recursively find all PNG files in the current directory. In this case, the ** pattern matches the current directory and all subdirectories.

You can also use the ** pattern for any directory, not just the current one. For example, /media/**/*.png would match all PNGs in media and it's subdirectories.

# #[macro_use]
# extern crate error_chain;
extern crate glob;

use glob::glob;
#
# error_chain! {
#     foreign_links {
#         Glob(glob::GlobError);
#         Pattern(glob::PatternError);
#     }
# }

fn run() -> Result<()> {
    for entry in glob("**/*.png")? {
        println!("{}", entry?.display());
    }

    Ok(())
}
#
# quick_main!(run);

Find all files with given pattern ignoring filename case.

glob-badge cat-filesystem-badge

Find all image files in the /media/ directory matching the img_[0-9]*.png pattern.

A custom MatchOptions struct is passed to the glob_with function making the glob pattern case insensitive while keeping the other options Default.

# #[macro_use]
# extern crate error_chain;
extern crate glob;

use glob::{glob_with, MatchOptions};
#
# error_chain! {
#     foreign_links {
#         Glob(glob::GlobError);
#         Pattern(glob::PatternError);
#     }
# }

fn run() -> Result<()> {
    let options = MatchOptions {
        case_sensitive: false,
        ..Default::default()
    };

    for entry in glob_with("/media/img_[0-9]*.png", &options)? {
        println!("{}", entry?.display());
    }

    Ok(())
}
#
# quick_main!(run);

Parse and increment a version string.

semver-badge cat-config-badge

Constructs a semver::Version from a string literal using Version::parse, then increments it by patch, minor, and major version number one by one.

Note that in accordance with the Semantic Versioning Specification, incrementing the minor version number resets the patch version number to 0 and incrementing the major version number resets both the minor and patch version numbers to 0.

# #[macro_use]
# extern crate error_chain;
extern crate semver;

use semver::Version;
#
# error_chain! {
#     foreign_links {
#         SemVer(semver::SemVerError);
#     }
# }

fn run() -> Result<()> {
    let mut parsed_version = Version::parse("0.2.6")?;

    assert_eq!(
        parsed_version,
        Version {
            major: 0,
            minor: 2,
            patch: 6,
            pre: vec![],
            build: vec![],
        }
    );

    parsed_version.increment_patch();
    assert_eq!(parsed_version.to_string(), "0.2.7");
    println!("New patch release: v{}", parsed_version);

    parsed_version.increment_minor();
    assert_eq!(parsed_version.to_string(), "0.3.0");
    println!("New minor release: v{}", parsed_version);

    parsed_version.increment_major();
    assert_eq!(parsed_version.to_string(), "1.0.0");
    println!("New major release: v{}", parsed_version);

    Ok(())
}
#
# quick_main!(run);

Parse a complex version string.

semver-badge cat-config-badge

Constructs a semver::Version from a complex version string using Version::parse. The string contains pre-release and build metadata as defined in the Semantic Versioning Specification.

Note that, in accordance with the Specification, build metadata is parsed but not considered when comparing versions. In other words, two versions may be equal even if their build strings differ.

# #[macro_use]
# extern crate error_chain;
extern crate semver;

use semver::{Identifier, Version};
#
# error_chain! {
#     foreign_links {
#         SemVer(semver::SemVerError);
#     }
# }

fn run() -> Result<()> {
    let version_str = "1.0.49-125+g72ee7853";
    let parsed_version = Version::parse(version_str)?;

    assert_eq!(
        parsed_version,
        Version {
            major: 1,
            minor: 0,
            patch: 49,
            pre: vec![Identifier::Numeric(125)],
            build: vec![],
        }
    );
    assert_eq!(
        parsed_version.build,
        vec![Identifier::AlphaNumeric(String::from("g72ee7853"))]
    );

    let serialized_version = parsed_version.to_string();
    assert_eq!(&serialized_version, version_str);

    Ok(())
}
#
# quick_main!(run);

Check if given version is pre-release.

semver-badge cat-config-badge

Given two versions, we assert (by using is_prerelease) that one is pre-release and that the other is not.

# #[macro_use]
# extern crate error_chain;
extern crate semver;

use semver::Version;
#
# error_chain! {
#     foreign_links {
#         SemVer(semver::SemVerError);
#     }
# }

fn run() -> Result<()> {
    let version_1 = Version::parse("1.0.0-alpha")?;
    let version_2 = Version::parse("1.0.0")?;

    assert!(version_1.is_prerelease());
    assert!(!version_2.is_prerelease());

    Ok(())
}
#
# quick_main!(run);

Find the latest version satisfying given range

semver-badge cat-config-badge

Given a list of version &strs, finds the latest semver::Version that satisfying a given semver::VersionReq using VersionReq::matches.

# #[macro_use]
# extern crate error_chain;
extern crate semver;

use semver::{Version, VersionReq};
#
# error_chain! {
#     foreign_links {
#         SemVer(semver::SemVerError);
#         SemVerReq(semver::ReqParseError);
#     }
# }

fn find_max_matching_version<'a, I>(version_req_str: &str, iterable: I) -> Result<Option<Version>>
where
    I: IntoIterator<Item = &'a str>,
{
    let vreq = VersionReq::parse(version_req_str)?;

    Ok(
        iterable
            .into_iter()
            .filter_map(|s| Version::parse(s).ok())
            .filter(|s| vreq.matches(s))
            .max(),
    )
}

fn run() -> Result<()> {
    assert_eq!(
        find_max_matching_version("<= 1.0.0", vec!["0.9.0", "1.0.0", "1.0.1"])?,
        Some(Version::parse("1.0.0")?)
    );

    // Shows Semver precedence for pre-release tags
    assert_eq!(
        find_max_matching_version(
            ">1.2.3-alpha.3",
            vec![
                "1.2.3-alpha.3",
                "1.2.3-alpha.4",
                "1.2.3-alpha.10",
                "1.2.3-beta.4",
                "3.4.5-alpha.9",
            ]
        )?,
        Some(Version::parse("1.2.3-beta.4")?)
    );

    Ok(())
}
#
# quick_main!(run);

Check external command version for compatibility

semver-badge cat-text-processing-badge cat-os-badge

Runs git --version using Command, then parses the version number into a semver::Version using Version::parse. A semver::VersionReq is used to compare the parsed version to a minimum version requirement.

# #[macro_use]
# extern crate error_chain;
extern crate semver;

use std::process::Command;
use semver::{Version, VersionReq};
#
# error_chain! {
#     foreign_links {
#         Io(std::io::Error);
#         Utf8(std::string::FromUtf8Error);
#         SemVer(semver::SemVerError);
#         SemVerReq(semver::ReqParseError);
#     }
# }

fn run() -> Result<()> {
    let version_constraint = "> 1.12.0";
    let version_test = VersionReq::parse(version_constraint)?;
    let output = Command::new("git").arg("--version").output()?;

    if !output.status.success() {
        bail!("Command executed with failing error code");
    }

    let stdout = String::from_utf8(output.stdout)?;
    // `git --version` output: "git version x.y.z"
    let version = stdout.split(" ").last().ok_or_else(|| {
        "Invalid command output"
    })?;
    let parsed_version = Version::parse(version)?;

    if !version_test.matches(&parsed_version) {
        bail!("Command version lower than minimum supported version (found {}, need {})",
            parsed_version, version_constraint);
    }

    Ok(())
}
#
# quick_main!(run);

Logging

Recipe Crates Categories
Log a debug message to the console log-badge env_logger-badge cat-debugging-badge
Log an error message to the console log-badge env_logger-badge cat-debugging-badge
Enable log levels per module log-badge env_logger-badge cat-debugging-badge
Log to stdout instead of stderr log-badge env_logger-badge cat-debugging-badge
Log messages with a custom logger log-badge cat-debugging-badge
Use a custom environment variable to set up logging log-badge env_logger-badge cat-debugging-badge
Include timestamp in log messages log-badge env_logger-badge chrono-badge cat-debugging-badge
Log to the Unix syslog log-badge syslog-badge cat-debugging-badge
Log messages to a custom location log-badge log4rs-badge cat-debugging-badge

Log a debug message to the console

log-badge env_logger-badge cat-debugging-badge

The log crate provides logging utilities. The env_logger crate configures logging via an environment variable.

#[macro_use]
extern crate log;
extern crate env_logger;

fn execute_query(query: &str) {
    debug!("Executing query: {}", query);

    // then do the thing
}

fn main() {
    env_logger::init();

    execute_query("DROP TABLE students");
}

If you run this code, you'll notice that no output is printed. By default, the log level is error, and any lower levels are dropped.

We can change that easily by setting the RUST_LOG environment variable:

$ RUST_LOG=debug cargo run

After running this, you'll likely see a pile of logs from cargo, as well as the following line at the very end of the output:

DEBUG:main: Executing query: DROP TABLE students

Log an error message to the console

log-badge env_logger-badge cat-debugging-badge

#[macro_use]
extern crate log;
extern crate env_logger;
#
# #[macro_use]
# extern crate error_chain;
#
# error_chain! {
#     foreign_links {
#         SetLogger(log::SetLoggerError);
#     }
# }

fn execute_query(_query: &str) -> Result<()> {
    // Do the thing, or maybe not

    bail!("I'm afraid I can't do that")
}

fn run() -> Result<()> {
    env_logger::init();

    let response = execute_query("DROP TABLE students");
    if let Err(err) = response {
        // Log the error message and continue
        error!("Failed to execute query: {}", err);
    }

    Ok(())
}
#
# quick_main!(run);

Run this code with cargo run and you should see the following line:

DEBUG:main: Failed to execute query: I'm afraid I can't do that

Enable log levels per module

log-badge env_logger-badge cat-debugging-badge

Creates two modules foo and nested foo::bar with logging directives controlled separately with RUST_LOG environmental variable.

#[macro_use]
extern crate log;
extern crate env_logger;

mod foo {
    mod bar {
        pub fn run() {
            warn!("[bar] warn");
            info!("[bar] info");
            debug!("[bar] debug");
        }
    }

    pub fn run() {
        warn!("[foo] warn");
        info!("[foo] info");
        debug!("[foo] debug");
        bar::run();
    }
}

fn main() {
    env_logger::init();
    warn!("[root] warn");
    info!("[root] info");
    debug!("[root] debug");
    foo::run();
}

env_logger output is controlled by RUST_LOG environmental variable on per module basis with comma separated entries in format path::to::module=log_level. Running the test application as follows:

RUST_LOG="warn,test::foo=info,test::foo::bar=debug" ./test

Sets the default log::Level to warn, module's foo and module's foo::bar respectively to info and debug. The output is:

WARN:test: [root] warn
WARN:test::foo: [foo] warn
INFO:test::foo: [foo] info
WARN:test::foo::bar: [bar] warn
INFO:test::foo::bar: [bar] info
DEBUG:test::foo::bar: [bar] debug

Log to stdout instead of stderr

log-badge env_logger-badge cat-debugging-badge

Creates a custom logger configuration using the Builder::target to set the target of the log output to Target::Stdout.

#[macro_use]
extern crate log;
extern crate env_logger;

use env_logger::{Builder, Target};

fn main() {
    Builder::new()
        .target(Target::Stdout)
        .init();

    error!("This error has been printed to Stdout");
}

Log messages with a custom logger

log-badge cat-debugging-badge

Implements a custom logger ConsoleLogger which prints to stdout. In order to use the logging macros, ConsoleLogger implements the log::Log trait and has to be installed via log::set_logger.

# #[macro_use]
# extern crate error_chain;
#[macro_use]
extern crate log;

use log::{Record, Level, Metadata, LevelFilter};

static CONSOLE_LOGGER: ConsoleLogger = ConsoleLogger;

struct ConsoleLogger;

impl log::Log for ConsoleLogger {
    fn enabled(&self, metadata: &Metadata) -> bool {
        metadata.level() <= Level::Info
    }

    fn log(&self, record: &Record) {
        if self.enabled(record.metadata()) {
            println!("Rust says: {} - {}", record.level(), record.args());
        }
    }

    fn flush(&self) {}
}
#
# error_chain! {
#     foreign_links {
#         SetLogger(log::SetLoggerError);
#     }
# }

fn run() -> Result<()> {
    log::set_logger(&CONSOLE_LOGGER)?;
    log::set_max_level(LevelFilter::Info);

    info!("hello log");
    warn!("warning");
    error!("oops");
    Ok(())
}
#
# quick_main!(run);

Use a custom environment variable to set up logging

log-badge env_logger-badge cat-debugging-badge

Logging is configured with Builder.

Builder::parse parses MY_APP_LOG environmental variable contents in the form of RUST_LOG syntax. Then Builder::init initializes the logger. All these steps are normally done internally by env_logger::init.

#[macro_use]
extern crate log;
extern crate env_logger;

use std::env;
use env_logger::Builder;

fn main() {
    Builder::new()
        .parse(&env::var("MY_APP_LOG").unwrap_or_default())
        .init();

    info!("informational message");
    warn!("warning message");
    error!("this is an error {}", "message");
}

Include timestamp in log messages

log-badge env_logger-badge chrono-badge cat-debugging-badge

Creates a custom logger configuration with Builder. Each log entry calls Local::now to get the current DateTime in local timezone and uses DateTime::format with strftime::specifiers to format a timestamp used in the final log.

The example calls Builder::format to set a closure which writes each message text with timestamp, Record::level and body (Record::args).

#[macro_use]
extern crate log;
extern crate chrono;
extern crate env_logger;

use std::env;
use std::io::Write;
use chrono::Local;
use env_logger::Builder;

fn main() {
    Builder::new()
        .format(|buf, record| {
            write!(buf,
                "{} [{}] - {}",
                Local::now().format("%Y-%m-%dT%H:%M:%S"),
                record.level(),
                record.args()
            )
        })
        .parse(&env::var("MY_APP_LOG").unwrap_or_default())
        .init();

    warn!("warn");
    info!("info");
    debug!("debug");
}

Calling MY_APP_LOG="info" cargo run will result in similar output:

2017-05-22T21:57:06 [WARN] - warn
2017-05-22T21:57:06 [INFO] - info

Log to the Unix syslog

log-badge syslog-badge cat-debugging-badge

Logs messages to UNIX syslog. Initializes logger backend with syslog::init. syslog::Facility indicates type of program submitting log, log::LevelFilter denotes allowed log verbosity and Option<&str> holds optional application name.

# #![allow(unused_imports)]
# #[macro_use]
# extern crate error_chain;
#[macro_use]
extern crate log;
# #[cfg(target_os = "linux")]
extern crate syslog;

# #[cfg(target_os = "linux")]
use syslog::Facility;
#
# #[cfg(target_os = "linux")]
# error_chain! {
#     foreign_links {
#         SetLogger(syslog::Error);
#     }
# }

# #[cfg(target_os = "linux")]
fn run() -> Result<()> {
    syslog::init(Facility::LOG_USER,
                 log::LevelFilter::Debug,
                 Some("My app name"))?;
    debug!("this is a debug {}", "message");
    error!("this is an error!");
    Ok(())
}
#
# #[cfg(not(target_os = "linux"))]
# error_chain! {}
# #[cfg(not(target_os = "linux"))]
# fn run() -> Result<()> {
#     Ok(())
# }
#
# quick_main!(run);

Log messages to a custom location

log-badge log4rs-badge cat-debugging-badge

Configures log to be output into custom location with log4rs. log4rs can use either an external YAML file or a programmatically constructed configuration.

Firstly creates the log configuration with log4rs::append::file::FileAppender using a custom pattern from log4rs::encode::pattern.

Secondly assigns it to the log4rs::config::Config which has a root appender that uses the previously created logfile appender. Subsequently sets the default log::LevelFilter so that any logs with Info level or higher will be sent to the logger.

# #[macro_use]
# extern crate error_chain;
#[macro_use]
extern crate log;
extern crate log4rs;

use log::LevelFilter;
use log4rs::append::file::FileAppender;
use log4rs::encode::pattern::PatternEncoder;
use log4rs::config::{Appender, Config, Root};
#
# error_chain! {
#     foreign_links {
#         Io(std::io::Error);
#         LogConfig(log4rs::config::Errors);
#         SetLogger(log::SetLoggerError);
#     }
# }

fn run() -> Result<()> {
    let logfile = FileAppender::builder()
        .encoder(Box::new(PatternEncoder::new("{l} - {m}\n")))
        .build("log/output.log")?;

    let config = Config::builder()
        .appender(Appender::builder().build("logfile", Box::new(logfile)))
        .build(Root::builder()
                   .appender("logfile")
                   .build(LevelFilter::Info))?;

    log4rs::init_config(config)?;

    info!("Hello, world!");

    Ok(())
}
#
# quick_main!(run);

Build Time Tooling

This section covers "build-time" tooling, or code that is run prior to compiling a crate's source code. Conventionally, build-time code lives in a build.rs file and is commonly referred to as a "build script". Common use cases include rust code generation and compilation of bundled C/C++/asm code. See crates.io's documentation on the matter for more information.

Recipe Crates Categories
Compile and link statically to a bundled C library cc-badge cat-development-tools-badge
Compile and link statically to a bundled C++ library cc-badge cat-development-tools-badge
Compile a C library while setting custom defines cc-badge cat-development-tools-badge

Compile and link statically to a bundled C library

cc-badge cat-development-tools-badge

To accommodate scenarios where additional C, C++, or assembly is required in a project, the cc crate offers a simple api for compiling bundled C/C++/asm code into static libraries (.a) that can be statically linked to by rustc.

The following example has some bundled C code (src/hello.c) that will be used from rust. Before compiling our rust source code, the "build" file (build.rs) specified in Cargo.toml will run. Using the cc crate, a static library file will be produced (in this case, libhello.a, see compile docs) which can then be used from rust by declaring the external function signatures in an extern block.

Since the bundled C is very simple, only a single source file needs to be passed to cc::Build. For more complex build requirements, cc::Build offers a full suite of builder methods for specifying include paths and extra compiler flags.

Cargo.toml

[package]
...
build = "build.rs"

[build-dependencies]
cc = "1"

[dependencies]
error-chain = "0.11"

build.rs

extern crate cc;

fn main() {
    cc::Build::new()
        .file("src/hello.c")
        .compile("hello");   // outputs `libhello.a`
}

src/hello.c

#include <stdio.h>


void hello() {
    printf("Hello from C!\n");
}

void greet(const char* name) {
    printf("Hello, %s!\n", name);
}

src/main.rs

# #[macro_use] extern crate error_chain;
use std::ffi::CString;
use std::os::raw::c_char;
#
# error_chain! {
#     foreign_links {
#         NulError(::std::ffi::NulError);
#         Io(::std::io::Error);
#     }
# }
#
# fn prompt(s: &str) -> Result<String> {
#     use std::io::Write;
#     print!("{}", s);
#     std::io::stdout().flush()?;
#     let mut input = String::new();
#     std::io::stdin().read_line(&mut input)?;
#     Ok(input.trim().to_string())
# }

extern {
    fn hello();
    fn greet(name: *const c_char);
}

fn run() -> Result<()> {
    unsafe { hello() }
    let name = prompt("What's your name? ")?;
    let c_name = CString::new(name)?;
    unsafe { greet(c_name.as_ptr()) }
    Ok(())
}
#
# quick_main!(run);

Compile and link statically to a bundled C++ library

cc-badge cat-development-tools-badge

Linking a bundled C++ library is very similar to linking a bundled C library. The two core differences when compiling and statically linking a bundled C++ library are specifying a C++ compiler via the builder method cpp(true) and preventing name mangling by the C++ compiler by adding the extern "C" section at the top of our C++ source file.

Cargo.toml

[package]
...
build = "build.rs"

[build-dependencies]
cc = "1"

build.rs

extern crate cc;

fn main() {
    cc::Build::new()
        .cpp(true)
        .file("src/foo.cpp")
        .compile("foo");   
}

src/foo.cpp

extern "C" {
    int multiply(int x, int y);
}

int multiply(int x, int y) {
    return x*y;
}

src/main.rs

extern {
    fn multiply(x : i32, y : i32) -> i32;
}

fn main(){
    unsafe {
        println!("{}", multiply(5,7));
    }   
}

Compile a C library while setting custom defines

cc-badge cat-development-tools-badge

It is simple to build bundled C code with custom defines using cc::Build::define. It takes an Option value, so it is possible to create defines such as #define APP_NAME "foo" as well as #define WELCOME (pass None as the value for a value-less define). This example builds a bundled C file with dynamic defines set in build.rs and prints "Welcome to foo - version 1.0.2" when run. Cargo sets some environment variables which may be useful for some custom defines.

Cargo.toml

[package]
...
version = "1.0.2"
build = "build.rs"

[build-dependencies]
cc = "1"

build.rs

extern crate cc;

fn main() {
    cc::Build::new()
        .define("APP_NAME", "\"foo\"")
        .define("VERSION", format!("\"{}\"", env!("CARGO_PKG_VERSION")).as_str())
        .define("WELCOME", None)
        .file("src/foo.c")
        .compile("foo");
}

src/foo.c

#include <stdio.h>

void print_app_info() {
#ifdef WELCOME
    printf("Welcome to ");
#endif
    printf("%s - version %s\n", APP_NAME, VERSION);
}

src/main.rs

extern {
    fn print_app_info();
}

fn main(){
    unsafe {
        print_app_info();
    }   
}