Structured Data

Serialize and deserialize unstructured JSON

serde_json Encoding

The serde_json crate provides a from_str function to parse a &str of JSON.

Unstructured JSON can be parsed into a universal serde_json::Value type that is able to represent any valid JSON data.

The recipe below shows a &str of JSON being parsed. The expected value is declared using the json! macro.

use serde_json::json;
use serde_json::{Error, Value};

fn main() -> Result<(), Error> {
    let j = r#"{
                 "userid": 103609,
                 "verified": true,
                 "access_privileges": [
                   "user",
                   "admin"
                 ]
               }"#;

    let parsed: Value = serde_json::from_str(j)?;

    let expected = json!({
        "userid": 103609,
        "verified": true,
        "access_privileges": [
            "user",
            "admin"
        ]
    });

    assert_eq!(parsed, expected);

    Ok(())
}

Deserialize a TOML configuration file

toml serde Encoding

Parse some TOML into a universal toml::Value that is able to represent any valid TOML data.

This recipe requires the derive feature flag of serde to be enabled in Cargo.toml.

use toml::{Value, de::Error};

fn main() -> Result<(), Error> {
    let toml_content = r#"
          [package]
          name = "your_package"
          version = "0.1.0"
          authors = ["You! <you@example.org>"]

          [dependencies]
          serde = "1.0"
          "#;

    let package_info: Value = toml::from_str(toml_content)?;

    assert_eq!(package_info["dependencies"]["serde"].as_str(), Some("1.0"));
    assert_eq!(
        package_info["package"]["name"].as_str(),
        Some("your_package")
    );

    Ok(())
}

Parse TOML into your own structs using Serde.

use serde::Deserialize;

use std::collections::HashMap;
use toml::de::Error;

#[derive(Deserialize)]
struct Config {
    package: Package,
    dependencies: HashMap<String, String>,
}

#[derive(Deserialize)]
struct Package {
    name: String,
    version: String,
    authors: Vec<String>,
}

fn main() -> Result<(), Error> {
    let toml_content = r#"
          [package]
          name = "your_package"
          version = "0.1.0"
          authors = ["You! <you@example.org>"]

          [dependencies]
          serde = "1.0"
          "#;

    let package_info: Config = toml::from_str(toml_content)?;

    assert_eq!(package_info.package.name, "your_package");
    assert_eq!(package_info.package.version, "0.1.0");
    assert_eq!(package_info.package.authors, vec!["You! <you@example.org>"]);
    assert_eq!(package_info.dependencies["serde"], "1.0");

    Ok(())
}

Read and write integers in little-endian byte order

std Encoding

Reversing the significant bytes of structured data may be necessary when receiving information over the network, such that bytes received are from another system. The standard library covers this without any extra crate.

Every integer type has methods that convert to and from a byte array in a chosen order: u16::to_le_bytes and u16::from_le_bytes for little-endian, u16::to_be_bytes and u16::from_be_bytes for big-endian, and u16::to_ne_bytes for the endianness of the current platform. The arrays have a fixed size, so no allocation is needed to convert a single value.

A byte slice implements Read, so Read::read_exact fills each array from the front of the slice and advances it. If the input is too short, it returns an UnexpectedEof error instead of panicking.

use std::io::{Error, Read};

#[derive(Default, PartialEq, Debug)]
struct Payload {
    kind: u8,
    value: u16,
}

fn main() -> Result<(), Error> {
    let original_payload = Payload {
        kind: 1,
        value: 0x0203,
    };
    let encoded_bytes = encode(&original_payload);
    assert_eq!(encoded_bytes, [0x01, 0x03, 0x02]);
    let decoded_payload = decode(&encoded_bytes)?;
    assert_eq!(original_payload, decoded_payload);
    assert!(decode(&encoded_bytes[..2]).is_err());
    Ok(())
}

fn encode(payload: &Payload) -> Vec<u8> {
    let mut bytes = vec![payload.kind];
    bytes.extend_from_slice(&payload.value.to_le_bytes());
    bytes
}

fn decode(mut bytes: &[u8]) -> Result<Payload, Error> {
    let mut kind = [0; 1];
    bytes.read_exact(&mut kind)?;
    let mut value = [0; 2];
    bytes.read_exact(&mut value)?;
    Ok(Payload {
        kind: kind[0],
        value: u16::from_le_bytes(value),
    })
}

The byteorder crate remains useful for streams. Its ReadBytesExt and WriteBytesExt traits read and write integers directly on any Read or Write without a temporary array, and it can convert whole slices of integers at once. For a handful of fixed-size fields, the standard library methods above are enough.