Description

Compile solves the problem of information overload. Instead of visiting dozens of news sites, users get feeds from multiple sources and categories aggregated, compiled into highlights and delivered by email twice a day. Technically, the challenge was reliability: continuously processing hundreds of feeds of varying quality and format, deduplicating content, and guaranteeing that scheduled sends go out on time with no manual intervention.

Key Features

• Multi-Source Aggregation: Automated collection of RSS feeds from global sources, with normalisation of content arriving in heterogeneous formats.

• Automated Digest: Compiled summaries sent twice daily to registered users, with highlights and direct links to original sources.

• Category Organisation: Content classified by topic, letting users follow only the areas they care about.

• Subscriber Management: Registration, category preferences and unsubscribe handling in line with email best practice.

Technologies Used

• Backend: Laravel with scheduled tasks (cron), processing queues and resilient handling of unavailable feeds.

• Feed Processing: RSS parser with content normalisation, article deduplication and metadata extraction.

• Database: Postgres with indexing optimised for querying large volumes of articles by date and category.

• Email Delivery: Bulk sending system with responsive templates and deliverability monitoring.

• Infrastructure: Cloud hosting with supervised background workers and execution monitoring for scheduled tasks.

Results and Impact

• Fully Automated: The platform runs with zero daily manual intervention collection, compilation and delivery all operate autonomously.

• Sources Aggregated: +23 sources processed across 4 categories.