https://marley.io/posts/ Recent content in Posts on marley.io Hugo -- 0.145.0 en-us Thu, 24 Apr 2025 00:00:00 +0000 https://marley.io/posts/rust-fts/ Thu, 24 Apr 2025 00:00:00 +0000 https://marley.io/posts/rust-fts/ I built a full-text search engine in Rust using Tantivy to ingest and index Brazil’s Diário Oficial da União–the federal government’s daily journal. It scans new publications for key phrases and sends real-time email alerts. https://marley.io/posts/go-s3-hasher/ Tue, 13 Aug 2024 00:00:00 +0000 https://marley.io/posts/go-s3-hasher/ Revisiting the S3 Hasher, but this time in Go! We achieve similar performance to Rust, processing at just around 3 GB/s using Go’s built-in concurrency primitives. https://marley.io/posts/rust-concurrent-s3-hashing/ Tue, 30 Jul 2024 00:00:00 +0000 https://marley.io/posts/rust-concurrent-s3-hashing/ A high-speed S3 object hasher built with Rust, processing at up to 3 GB/s. We use a channel-based architecture to concurrently stream and hash S3 objects. We process a 12 TB dataset in just over an hour, demonstrating Rust’s capability for efficient, large-scale data processing.