FOP - Filter Orderer and Preener (Rust Edition)

A Rust port of the EasyList FOP tool for sorting and cleaning ad-blocking filter lists.

Features

• Filter sorting: Alphabetically sorts blocking rules and element hiding rules
• Domain combining: Merges rules with identical selectors/patterns but different domains
• Option normalization: Converts uBO-specific options to standard ABP format (can be disabled)
• Wildcard cleanup: Removes unnecessary wildcards from filters
• Validation: Removes invalid/overly-broad rules (TLD-only, too short, etc.)
• Git integration: Commit changes directly to repositories (can be disabled)
• easylist_adservers.txt validation: Ensures rules start with | or /
• :has-text() merging: Combines rules with same base selector into single regex
• Parallel processing: Uses all CPU cores for faster processing via Rayon

Extended Syntax Support

FOP preserves extended filter syntax from various adblockers:

uBlock Origin

• Scriptlet injection: ##+js(...), ##^script:has-text(...)
• Procedural cosmetics: :has-text(), :has(), :upward(), :remove(), :style(), :matches-css(), :xpath(), :remove-attr(), :remove-class(), :watch-attr(), :min-text-length(), :others()
• Network options: redirect=, redirect-rule=, removeparam=, denyallow=, replace=, header=, permissions=, to=, from=, method=
• Regex domain rules: /regex/##+js(...)

AdGuard

• Scriptlet injection: #%#//scriptlet(...)
• CSS injection: #$#body { ... }
• Exceptions: #@%#, #@$#

Adblock Plus

• Extended selectors: :-abp-has(), :-abp-contains(), :-abp-properties()
• Action syntax: {remove: true;}, {height:...}, {display:...}
• Snippets: #$#hide-if-contains, #$#simulate-mouse-event

Speed Comparison (FOP.py vs FOP Rust)

Installation

npm install -g fop-cli

Building from source (optional)

Requires Rust 1.80+ (https://rustup.rs)

cd fop-rs
cargo build --release
cp target/release/fop /usr/local/bin/  # or add to PATH

Usage

# Sort filters in current directory (with commit prompt)
fop

# Sort filters in a specific directory
fop /path/to/easylist

# Sort without commit prompt (like FOP-nocommit.py)
fop --no-commit /path/to/easylist
fop -n .

# Sort without converting uBO options to ABP format
fop --no-ubo-convert /path/to/ublock-filters

# Sort multiple directories
fop -n ~/easylist ~/easyprivacy ~/fanboy-addon

Command Line Options

Option	Description
-n, --no-commit	Just sort files, skip Git commit prompts
--just-sort	Alias for --no-commit
--no-ubo-convert	Skip uBO to ABP option conversion (keep xhr, 3p, 1p, etc.)
--no-msg-check	Skip commit message format validation (M:/A:/P:)
--disable-ignored	Disable hardcoded ignored files and folders for testing
--no-sort	Don't sort rules, just combine
--alt-sort	More correct sorting method
--localhost	Sort hosts file entries (0.0.0.0/127.0.0.1 domain)
--localhost-files=	Specific files to sort as localhost format (comma-separated)
--parse-adguard	Globally Parse AdGuard extended CSS (#$?#, #@$?#, $$, $@$)
--parse-adguard=FILE	Files to parse as AdGuard extended CSS (comma-separated)
--no-color	Disable colored output
--no-large-warning	Disable large change warning prompt
--backup	Create .backup files before modifying
--keep-empty-lines	Keep empty lines in output
--ignore-dot-domains	Don't skip rules without dot in domain
--ignorefiles=	Additional files to ignore (comma-separated, partial names)
--ignoredirs=	Additional directories to ignore (comma-separated, partial names)
--ignore-all-but=	Only process these files, ignore all others (comma-separated)
--file-extensions=	File extensions to process (default: .txt)
--comments=	Comment line prefixes (default: !)
--warning-output=	Output warnings to file instead of stderr
--git-message=	Git commit message (skip interactive prompt)
--history=	Predefined commit messages for arrow key selection (comma-separated)
--create-pr[=TITLE]	Create PR branch instead of committing to current branch
--git-pr-branch=NAME	Base branch for PR (default: auto-detect main/master)
--pr-show-changes	Include rule changes (combines, merges, typos) in PR body
--fix-typos	Fix cosmetic rule typos in all files during sort
--fix-typos-on-add	Check cosmetic rule typos in git additions before commit
--auto-fix	Auto-fix typos without prompting (use with --fix-typos-on-add)
--only-sort-changed	Only process files changed according to git
--check-banned-list=FILE	Check for banned domains in git additions
--auto-banned-remove	Auto-remove banned domains and commit
--ci	CI mode - exit with error code on failures (banned domains)
--rebase-on-fail	Auto rebase and retry push if it fails
--ignore-config	Ignore .fopconfig file, use only CLI args
--output	Output changed files with --changed suffix (no overwrite)
--check-file=FILE	Process a single file
--output-diff=FILE	Output changes as diff (no files modified)
--quiet	Limit console output, less verbose
--limited-quiet	Suppress directory listing only
--add-timestamp=FILES	Update timestamp for specific files only (comma-separated)
--validate-checksum=FILES	Validate checksum for specific files (exit 1 on failure)
--validate-checksum-and-fix=FILES	Validate and fix invalid checksums
--add-checksum=FILES	Add/update checksum for specific files (comma-separated)
--add-timestamp	Update timestamp in file header (Last Modified/Last Updated)
--config-file=	Custom config file path
--show-config	Show applied configuration and exit
--git-binary=<path>	Path to git binary (default: git in PATH)
--benchmark	Benchmark sorting performance (3 iterations, dry-run)
--abp-convert	Convert ABP extended selectors to uBO format
-h, --help	Show help message
-V, --version	Show version number

Configuration File

Create .fopconfig in your working directory or home directory:

# Skip commit prompt
no-commit = false

# Skip uBO to ABP option conversion
no-ubo-convert = false

# Skip commit message format validation
no-msg-check = false

# Skip sorting (only tidy and combine rules)
no-sort = false

# Alternative sorting method
alt-sort = false

# Sort hosts file entries
localhost = false

# Disable colored output
no-color = false

# Disable large change warning prompt
no-large-warning = false

# Create .backup files before modifying
backup = false

# Keep empty lines in output
keep-empty-lines = false

# Don't skip rules without dot in domain
ignore-dot-domains = false

# Comment line prefixes
comments = !

# Output warnings to file
warning-output =

# Additional files to ignore
ignorefiles = .json,.backup,.bak,.swp,.gz

# Additional directories to ignore
ignoredirs =

# File extensions to process
file-extensions = txt

# Create PR branch instead of committing
create-pr =

# Include rule changes in PR body
pr-show-changes = false

# Base branch for PR (default: auto-detect)
git-pr-branch =

# Fix cosmetic typos during sort
fix-typos = false

# Check typos in git additions
fix-typos-on-add = false

# Auto-fix without prompting
auto-fix = false

# Only sort git-changed files (skip unchanged)
only-sort-changed = false

# Path to banned domain list file
check-banned-list =

# Auto-remove banned domains and commit
auto-banned-remove = false

# CI mode - exit with error code on failures
ci = false

# Auto rebase and retry if push fails
rebase-on-fail = false

# Suppress most output (for CI)
quiet = false

# Users allowed to push directly when create-pr is enabled (comma-separated, case-insensitive)
direct-push-users =

# Predefined commit messages for arrow key selection (comma-separated)
# Use up/down arrows at commit prompt to cycle through these
history = A: ,P: ,M: Update,M: Cleanup,M: Sort,M: Adjust

Command line arguments override config file settings.

Platform Support

Pre-built Binaries

Platform	Binary	Optimization	Compatible Devices
Linux
x86_64	linux-x86_64	Baseline	All 64-bit Intel/AMD
x86_64	linux-x86_64-v3	AVX2	Intel Haswell+ / AMD Excavator+ (~2015+)
x86	linux-x86_32	Baseline	32-bit systems, older hardware
ARM64	linux-arm64	Baseline	Raspberry Pi 3/4/5, Orange Pi 3/4/5, all ARM64
ARM64	linux-arm64-n1	Neoverse N1	Pi 5, Orange Pi 5, AWS Graviton2+, Ampere Altra
RISC-V	linux-riscv64	Baseline	SiFive, StarFive VisionFive 2, Milk-V
macOS
Intel	macos-x86_64	Baseline	All Intel Macs
Apple Silicon	macos-arm64	Apple M1	M1, M2, M3, M4, M5 Macs
Windows
x86_64	windows-x86_64.exe	Baseline	All 64-bit Windows
x86_64	windows-x86_64-v3.exe	AVX2	Intel Haswell+ / AMD Excavator+ (~2015+)
x86	windows-x86_32.exe	Baseline	32-bit Windows
ARM64	windows-arm64-v2.exe	Cortex-A78	Surface Pro X, Snapdragon laptops

Which binary should I use?

Linux/Windows x86_64:

• Use -v3 for CPUs from ~2015+ (Haswell, Ryzen) - ~10-20% faster due to AVX2
• Use baseline if unsure or on older CPUs

Linux ARM64 (Raspberry Pi / Orange Pi):

• Use linux-arm64 for Pi 3, Pi 4, Orange Pi 3/4, or if unsure
• Use linux-arm64-n1 for Pi 5, Orange Pi 5, AWS Graviton2+ (~10-20% faster)

npm install downloads baseline binaries for maximum compatibility. Optimized versions are available from GitHub Releases.

Build Details

Binary	Target	RUSTFLAGS
linux-x86_64	Native	-
linux-x86_64-v3	Native	-C target-cpu=x86-64-v3
linux-x86_32	i686-unknown-linux-gnu	-
linux-arm64	aarch64-unknown-linux-gnu	-
linux-arm64-n1	aarch64-unknown-linux-gnu	-C target-cpu=neoverse-n1
linux-riscv64	riscv64gc-unknown-linux-gnu	-
macos-x86_64	x86_64-apple-darwin	-
macos-arm64	aarch64-apple-darwin	-C target-cpu=apple-m1
windows-x86_64.exe	x86_64-pc-windows-gnu	-
windows-x86_64-v3.exe	x86_64-pc-windows-gnu	-C target-cpu=x86-64-v3
windows-x86_32.exe	i686-pc-windows-gnu	-
windows-arm64-v2.exe	aarch64-pc-windows-msvc	-C target-cpu=cortex-a78

Unsupported platform? FOP.rs builds from source on any platform with Rust 1.80+:

cargo build --release

Migrating from Python FOP

This Rust version is a drop-in replacement for both FOP.py and FOP-nocommit.py:

Python	Rust Equivalent
python3 FOP.py	fop
python3 FOP-nocommit.py	fop --no-commit or fop -n

Performance

The Rust version is significantly faster than Python FOP due to:

• Compiled native code
• Parallel file processing with Rayon
• Optimized regex handling
• Efficient memory management

Windows PowerShell Note

If you see UnauthorizedAccess or PSSecurityException when running fop in PowerShell, run this once to fix permanently:

Set-ExecutionPolicy RemoteSigned -Scope CurrentUser

Alternatively, use Git Bash, MINGW64, or WSL.

Security Warnings

Pre-built binaries aren't code-signed, so your OS may show a warning on first run.

macOS

If blocked by Gatekeeper ("cannot be opened because the developer cannot be verified"):

xattr -d com.apple.quarantine ./fop-mac

Windows

If SmartScreen blocks the file:

1. Click "More info"
2. Click "Run anyway"

Alternative: Install via npm (npm install -g fop-cli) to avoid these warnings entirely.

License

GPL-3.0 (same as original Python FOP)

Credits

• Original Python FOP by Michael (EasyList project)
• Rust port maintains feature parity with Python version 3.9