xiiregexbuilder

FPGA-Accelerated Regular Expression Matching Engine
commit 3aaf0d4787e0ccdd4ace8176bdb762f29fd014fc
parent f4f8ccdf923532ea7e9a579c6ad460a33c6fc08c
Author: Vishrut Gurrala <maydayv7@gmail.com>
Date:   Tue, 24 Mar 2026 20:17:08 +0530

Week 1: Add README and Use cases (Part 2: Core logic)

Diffstat:
MREADME.md | 71+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Mdocs/usecases.md | 15+++++++++++++++
2 files changed, 86 insertions(+), 0 deletions(-)

diff --git a/README.md b/README.md @@ -51,3 +51,74 @@ Every time the NFA engine finishes a string (newline received), the FPGA sends o ```text MATCH=<N-bit binary> BYTES=<8 hex digits> HITS=<4hex per regex, comma-separated>\r\n +``` + +Example (6 regexes, regexes 0 and 2 matched, 71 total bytes processed): + +```text +MATCH=000101 BYTES=00000047 HITS=0003,0001,0012,0000,0000,0000 +``` + +Send `?` at any time to query the current counters without feeding any character to the NFA. + +### Hardware Counters + +- `byte_count [31:0]` — total bytes fed into the NFA engine since the last reset. +- `match_count_k [15:0]` — cumulative match events for regex _k_ (independent counter per regex). + +## 5. System Scope + +### Minimum System (Commitment) + +- C++ Verilog Emitter compiling basic literal and concatenation patterns (e.g., abc). +- At least one generated Verilog FSM successfully simulated and executing correctly on the FPGA. + +### Goal System — **ACHIEVED** + +- Full compiler support for complex operators (`*`, `+`, `?`, `|`, `.`) via Glushkov's epsilon-free construction. +- Integration of up to 16 parallel FSM modules passing the C++ golden reference testbenches. +- UART host-side controller: host application sends arbitrary test strings and receives match bitmasks over serial in real time. +- **UART Transmitter** (`uart_tx.v`): dedicated TX module with a serializer state machine that formats and sends structured ASCII result packets. +- **Input FIFO Buffer** (`uart_rx_fifo.v`): 16-byte circular FIFO decouples the UART receiver from the NFA engine FSM, eliminating byte-drop risk. +- **Hardware Counters**: `byte_count` and per-regex `match_count` registers, queryable via UART or the Python TUI. + +### Stretch Goal + +- Bounded quantifiers `{m,n}`: compiler and hardware support for repetition counts. +- Live FIX message demo: stream a recorded FIX log from the host and demonstrate real-time field extraction. + +## 6. Design Goals + +- **Latency**: 1 clock cycle per ASCII character; match output registered on the cycle immediately following `end_of_str` assertion. +- **Throughput**: 1 byte per clock cycle continuously (100 MB/s at 100 MHz). +- **Resource usage**: Efficient one-hot state encoding (exactly N+1 flip-flops per FSM). +- **Correctness**: 100% full-match accuracy against the C++ `std::regex` golden reference. + +## 7. Quick Start + +### Build the C++ compiler and generate Verilog + +```bash +make run # builds regex_builder, runs it on inputs/regexes.txt +``` + +### Simulate in Vivado + +```bash +make sim # xvlog + xelab + xsim +``` + +### Synthesise and program the Nexys A7 + +```bash +make synth # runs synth.tcl through Vivado batch mode +make program # programs the attached FPGA +``` + +### Launch the Python TUI + +```bash +pip install pyserial rich +python tui.py --port /dev/ttyUSB0 --regexes inputs/regexes.txt +# On Windows: python tui.py --port COM3 --regexes inputs/regexes.txt +``` diff --git a/docs/usecases.md b/docs/usecases.md @@ -8,3 +8,18 @@ Modern financial markets generate enormous volumes of structured text — orders ## Key Use Cases in Quantitative Finance +**FIX Protocol Parsing.** Every order, cancel, and fill in electronic trading is an ASCII FIX message. Filtering and validating millions of these per second in hardware means routing decisions are made before the message reaches software — shaving off critical microseconds. + +**Market Data Feed Filtering.** Exchanges blast full order book updates at millions of events per second. Running regex matches in hardware lets firms discard irrelevant instruments at line rate, so the CPU only processes what the strategy actually needs. + +**Tick Data Pattern Detection.** Price movement sequences — bid size spikes, spread collapses, momentum signals — can be encoded as regex patterns over a tokenised tick stream. Hardware detection latency is measured in nanoseconds, not microseconds. + +**News and Sentiment Feeds.** Scanning incoming Reuters or Bloomberg text for company names, earnings figures, or central bank language is a multi-regex problem. An FPGA can flag a relevant news item before the full message has even arrived. + +**Trade Surveillance and Compliance.** Patterns like spoofing, layering, and wash trading are detectable as sequences in live order flow. Hardware matching enables real-time compliance checks on the live feed rather than retrospective analysis on stored logs. + +--- + +## Why FPGA and Not Just Faster Software? + +The fundamental advantage is parallelism. A software engine checks N regexes sequentially — throughput degrades linearly as N grows. An FPGA runs all N FSMs in the same clock cycle, so throughput is flat regardless of how many patterns are active. At scale, no amount of SIMD optimisation closes that gap.