🚀 Getting Started with Binspector

Binspector is a library to create declarative binary file formats definitions. It provides the tools to encode and decode binary file based on these definitions.

It takes heavy inspiration from C structs. which provides declarative, simple and elegant way to describe a binary data structures.

struct {
    uint8_t foo;
    uint16_t bar;
}
Copy

Since Typescript lacks built-in structures to describe integer length, Binspector introduces decorators used alongside class properties to mimic that behaviour:

class {
    @Uint8 foo
    @Uint16 bar
}
Copy

Using decorators to define binary structures offers several advantages:

• Declarative & Readable – The binary format is described naturally alongside class properties. You define the binary format in the same place as the Typescript class.
• No Duplicate Type Definitions – The binary definition doubles as the type definition, leveraging TypeScript's type system.
• Seamless Integration – Methods that process binary data can be written alongside the format definition.

❓ Why a TypeScript library

Using JS or one of its dialect to perform lower level operation on binary files may not be the most obvious choice. Working with a language such as TypeScript makes it easy for the code to run on webpages, and while a lot of tools are available online to modify or analyze binary files they often threat the file in their backend. Using a TypeScript library allows us to do all of this entirely in the frontend making you able to provide webapp that threat binary data running on Github Pages.

Other library exists for binary parsing in TS, but Binspector prioritzes declarative a declarative syntax and serialization support.

⚙️ How does it works

Each decorator you use on top of a property or class will store metadata providing information on how read and write the binary file definition.

📂 Categories of Decorators

The decorators are sorted into different categories:

🔍 The binary definition reading

Let's define a simple definition with Binspector.

class Protocol {
    @Uint8 foo;
    @Uint16 bar;
}
Copy

To parse the content of a file formatted as the Protocol definition you will use the following code.

import { BinaryReader, binread } from 'binspector'
import * as fs from 'node:fs'
import * as path from 'node:path'

const data = fs.readFileSync(path.join(import.meta.dirname, 'file.bin'))
const protocol = binread(data, Protocol)
Copy

From an high level point of view the binread function will first check whether the structure is a bitfield or a relation type definition.

stateDiagram-v2
state if_state <<choice>>
start: Reading a type definition
state start {
    PreClass: Execute __PreClass__ functions
    [*] --> PreClass
    PreClass --> if_state
    if_state --> BitField: is a __bitfield__
    if_state --> Relation : is a __relation type definition__
    state Relation {
        ReadRel: Read each property of the _relation type definition_.
    }
    state BitField {
        ReadBF: Read the BitField type definition based on the sized computed from the property definitions.
    }
    BitField --> PostClass
    Relation --> PostClass
    PostClass: Execute __PostClass__ functions
    PostClass --> [*]
}

As described above relation type definition can take advantage of a lot of decorators categories to describe how each property of your definition is read. The binread function will threat those categories sequentially:

flowchart TB
 subgraph s1[For each properties]
 direction TB
 PreOperation[__Pre__ property reading operations] --> Condition
 click PreOperation "/binspector/modules/PrePost.html" "Documentation for 'Pre' type decorators"
 Condition[__Condition__ get the definitive subtype to read based on current state] --> s2
 click Condition "/binspector/modules/Condition.html" "Documentation for 'Condtion' type decorators"
 subgraph s2[Reading subtype]
 Controller[__Controller__ decides when to stop reading the subtype based on a set of arbitrary conditions] --> TypeReading[Read __Relation__ or __Primitive__]
 click Controller "/binspector/modules/Controller.html" "Documentation for 'Controller' type decorators"
 click TypeReading "/binspector/modules/Primitive.html" "Documentation for 'Primitive' type decorators"
 end
 TypeReading --> Controller
 s2 --> Transform[__Transform__ the value we read into something else]
 click Transform "/binspector/modules/Transformer.html" "Documentation for 'Transformer' type decorators"
 Transform --> Validate[__Validate__ the final value]
 click Validate "/binspector/modules/Validator.html" "Documentation for 'Validator' type decorators"
 Validate --> PostOperation[__Post__ property reading operations]
 click PostOperation "/binspector/modules/PrePost.html" "Documentation for 'Post' type decorators"
 end
 PostOperation -->  A@{ shape: framed-circle, label: "Stop" }

✍ The binary definition writing

The following code snippets shows how to serialize an object into a binary buffer based on a Binspector definition. That buffer is then saved into a file.

import { BinaryWriter, binwrite } from 'binspector'
import { promises as fs } from 'fs'
import path from 'path'

const obj = {
    foo: 0x00,
    bar: 0x01
}

const protocol = binwrite(obj, Protocol)

const buf = protocol.buffer // <= ArrayBuffer(...)

await fs.appendFile(path.join(__dirname, 'proto.bin'), new Uint8Array(buf));
Copy

From an high level point of view the writing procedure almost reverse the steps done during the reading phase.

flowchart TB
 subgraph s1[For each properties]
 direction TB
 PreOperation[__Pre__ property reading operations] --> Condition
 click PreOperation "/binspector/modules/PrePost.html" "Documentation for 'Pre' type decorators"
 Condition[__Condition__ resolve the subtype] --> Transform
 click Condition "/binspector/modules/Condition.html" "Documentation for 'Condtion' type decorators"
 Transform[Scoped __Transformer__ transform the final value into the writable form] --> Flatten
 click Transform "/binspector/modules/Transformer.html" "Documentation for 'Transformer' type decorators"
 Flatten[__Flatten__ the array if value is a matrix]-->  s2
 subgraph s2[Writing subtype]
 Transform2[Lower level scoped __Transform__ on single element] --> TypeWrite[Write __Relation__ or __Primitive__]
 click TypeWrite "/binspector/modules/Primitive.html" "Documentation for 'Primitive' type decorators"
 TypeWrite --> Transform2
 end
 s2 --> PostOperation[__Post__ writing operation]
 click PostOperation "/binspector/modules/PrePost.html" "Documentation for 'Post' type decorators"
 end
 PostOperation -->  A@{ shape: framed-circle, label: "Stop" }

🌐 Using Binspector in the browser

Considering the following tag in the webpage.

Copy

A file input that will read the content of the file when sent to the webpage and a button that will download a serialized version of the object sent to the webpage.

The following code shows how to handle the content with Binspector.

import { BinaryReader, binread } from 'binspector'
import * as fs from 'node:fs'
import * as path from 'node:path'

let protocol = undefined

const fileInput = document.getElementById("file-input")
fileInput.addEventListener("change", handleFileInput)

const fileDownload = document.getElementById("file-download")
fileDownload.addEventListener("click", handleFileDownload)

function handleFileInput(event) {
  const file = event.target.files[0]
  file.arraybuffer().then((arr) => {
      protocol = binread(arr, Protocol)
  })
}

function handleFileDownload(event) {
    if (protocol !== undefined) {
        const _protocol = binwrite(protocol, Protocol)

        const blob = new Blob([_protocol.buffer])
        const url = URL.createObjectURL(blob)
        window.open(url)
    }
}
Copy