commit c622612af22a288f67797d3efd77db0bd1594dc5
parent 7ce00ce8fe35c9aea464761be50a601e144d1615
Author: yash modi <yash3108m@gmail.com>
Date: Sun, 12 Apr 2026 10:00:00 +0530
Week 5: Core processor logic and parallel engine implementation (Part 1: Initial definitions)
Diffstat:
2 files changed, 78 insertions(+), 24 deletions(-)
diff --git a/docs/details.md b/docs/details.md
@@ -148,8 +148,6 @@ A fully functional testbench (`tb_top.v`) is generated:
This stage connects the Verilog regex engine to the physical FPGA I/O: a USB-UART serial link for bidirectional communication with a host PC. Three new hardware modules are added.
----
-
### 4.1 UART Transmitter (`uart_tx.v`)
A standard 8-N-1 UART transmitter with a 4-state FSM:
@@ -221,35 +219,51 @@ This sub-FSM runs concurrently with the control FSM, which simply waits in `S_TX
Both are cleared by asserting `rst_btn`. Values are transmitted as part of every response packet.
-#### Response Packet Format
+---
-```
-MATCH=<N-bit binary> BYTES=<8 hex digits> HITS=<4 hex per regex, comma-separated>\r\n
-```
+## Stage 5 — Processor-based Regex Engine
-The `build_response` Verilog task constructs this string in a 128-byte register array at synthesis time (combinational logic mapped to LUTs), then stores it in `tx_buf`. No block RAM is required for the TX buffer.
+The processor-based engine (`processor/` directory) provides a dynamic alternative to the static Verilog FSMs. Instead of synthesising a new circuit for each regex, we use a custom Soft-Processor that executes "Regex Instructions" loaded into memory.
----
+### 5.1 Regex CPU Architecture
-### 4.4 Python TUI (`tui.py`)
+The Regex CPU is a specialized processor optimized for NFA simulation:
-A terminal user interface for interactive testing from any host PC.
+- **Instruction Set:** Custom 32-bit instructions (CHAR, SPLIT, JMP, MATCH, ANY).
+- **Instruction Memory:** 256-word × 32-bit memory (BRAM/LUTRAM inferred).
+- **State Representation:** A 256-bit wide `active_candidates` register represents the set of currently active NFA states.
+- **Execution Model:**
+ - **Character Match Phase:** Iterates through all active states and checks for a character match.
+ - **Epsilon Expansion Phase:** Iterates through active states to follow SPLIT and JMP transitions until only character-matching or terminal states remain active.
+ - **Terminal Phase:** Checks if any active state is a MATCH state at the end of the input string.
-**Dependencies:** `pip install pyserial rich`
+### 5.2 Glushkov Assembler Toolchain
-**Usage:**
+A Python-based compiler converts standard regular expressions into the processor's native machine code:
-```bash
-python tui.py --port /dev/ttyUSB0 --regexes inputs/regexes.txt
-```
+1. **`compile_regex.py`**:
+ - Parses regex into an AST.
+ - Computes Glushkov `first`, `last`, and `follow` sets.
+ - Generates a `SPLIT` chain to allow multiple regexes to run in parallel.
+ - Outputs an assembly file (`.rasm`).
+2. **`asm.py`**:
+ - Parses the `.rasm` file.
+ - Packs instruction fields into 32-bit binary words.
+ - Outputs a hex file (`imem.hex`) for FPGA memory initialization or runtime programming.
+
+### 5.3 Instruction Format
-The port is auto-detected if omitted (first USB-Serial device found).
+| Field | Bits | Description |
+| ------- | ------- | ------------------------------------------------ |
+| `char` | [31:24] | ASCII character to match (or 0 for epsilon/any). |
+| `next1` | [23:16] | Primary target PC for jump/split. |
+| `next2` | [15:8] | Secondary target PC for split. |
+| `mid` | [7:4] | Match ID (Regex index 0–15). |
+| `term` | [3] | Terminal bit (1 if this is a MATCH state). |
+| `any` | [0] | Wildcard bit (1 if this matches any character). |
-**Features:**
+### 5.4 Advantages of the Processor Approach
-- Live colour-coded match table: green `● MATCH` / dim red `○ —` per regex.
-- Running totals: total bytes processed and per-regex cumulative hit count.
-- Input history in the prompt.
-- Background reader thread handles FPGA responses without blocking the UI.
-- `?` query support: displays current counters without sending a test string.
-- Graceful exit on `q`, `quit`, or Ctrl-C.
+- **Runtime Flexibility:** Regexes can be updated by simply writing to the instruction memory over UART.
+- **Resource Efficiency:** Supports up to 16 complex regexes with a fixed amount of FPGA logic, regardless of regex complexity (up to 256 instructions).
+- **Deterministic Latency:** Fixed scan time of 256 cycles per character ensures predictable performance.
diff --git a/docs/specifications.md b/docs/specifications.md
@@ -448,7 +448,47 @@ MATCH=<N bits, MSB first> BYTES=<8 hex> HITS=<4 hex per regex, comma-sep>\r\n
---
-### 13.6 Timing Budget
+## 14. Stage 6 — Processor-based Dynamic Regex Engine
+
+### 14.1 Motivation
+
+While the static Verilog FSMs provide the highest possible performance and lowest latency, they require a full FPGA synthesis and implementation cycle to update the regex set. The Processor-based engine addresses this by providing a software-programmable NFA accelerator that can be updated in milliseconds via UART.
+
+### 14.2 Architecture Decisions
+
+| Decision | Choice |
+| --------------- | ------------------------------------------------------------- |
+| Parallelism | Bit-vector state representation (256-bit `active_candidates`) |
+| Instruction Set | Custom 5-instruction RISC (CHAR, SPLIT, JMP, MATCH, ANY) |
+| Throughput | 256 clock cycles per input character (fixed) |
+| Max Regexes | 16 independent patterns |
+| Max States | 256 total NFA states (instruction memory limit) |
+| Memory | Dual-port inferred BRAM for instruction storage |
+
+### 14.3 Instruction Encoding (32-bit)
+
+```text
+[31:24] char : ASCII literal to match (0 if not a character match)
+[23:16] next1 : Primary target PC for jump/split
+[15:08] next2 : Secondary target PC for split (0 if not used)
+[07:04] mid : Match ID (0-15) for reporting matches
+[03] term : Terminal bit (1 if this state triggers a match)
+[00] any : Wildcard bit (1 if this matches any character)
+```
+
+### 14.4 Execution Pipeline
+
+1. **IDLE**: Wait for `char_valid` or `start` or `end_of_str`.
+2. **START_INIT**: Activate PC 0 and transition to Epsilon Recurse.
+3. **CHAR_MATCH**: Scan all 256 instructions. If `active_candidates[pc]` is set and `char` matches `char_in`, set `next_set_buffer[next1/next2]`.
+4. **EPSILON_RECURSE**: Multi-pass scan (up to 64 passes). If `active_candidates[pc]` is an epsilon state (SPLIT/JMP), deactivate it and activate its targets. Continue until no more changes occur.
+5. **END_OF_STR**: Scan all active states. If any state has the `term` bit set, assert the corresponding bit in `match_bus[mid]`.
+
+### 14.5 Toolchain
+
+- **`compile_regex.py`**: A Glushkov-based compiler that generates a unified NFA for all input regexes and produces a `.rasm` assembly file.
+- **`asm.py`**: A simple assembler that converts `.rasm` into an FPGA-ready `.hex` file.
+- **`prog_fpga.py`**: A utility to stream the `.hex` file over UART to the processor's instruction memory at runtime.
At 100 MHz, one complete string-processing cycle from `\n` receipt to TX-complete takes: