s3

13 repositories

rclone/rclone

A unified command line for syncing dozens of cloud storage backends — Rclone syncs files across more than 70 cloud storage providers, from S3 and Google Drive to niche services like Cloudinary and Autonomys Auto Drive, using one consistent CLI. It supports encryption, FUSE mounting and WebDAV, making it a practical tool for backups and multi-cloud workflows.

GoAttention 29
★60,928
seaweedfs/seaweedfs

One Go binary unifying S3, POSIX, and Iceberg lakehouse storage — SeaweedFS combines an S3 object store, a POSIX file system, and a lakehouse with S3 Tables in a single 'weed' binary over shared data. It targets billions of files with O(1) disk access, horizontal scaling via extra volume servers, and transparent tiering to cloud storage.

Go🇺🇸 North AmericaAttention 41
★36,012
rustfs/rustfs

A Rust-built S3 alternative aimed at AI and big data workloads — RustFS is a distributed, S3-compatible object storage system written in Rust, designed to migrate from or coexist with MinIO and Ceph. It targets cloud-native and multi-cloud setups, with AI-native storage features for teams handling large-scale data pipelines.

Rust🇺🇸 North AmericaAttention 40
★35,945
floci-io/floci

A fully free, no-signup local AWS emulator for dev and CI — Floci emulates AWS services like EC2, ECS, S3, and SQS locally with no account, auth token, or feature gates required—just `docker compose up`. It targets developers and CI pipelines who want AWS-shaped testing without LocalStack's paid tiers, and includes a documented migration path from LocalStack.

Java🇺🇸 North AmericaAttention 61
★31,289
airbytehq/airbyte

Airbyte widens its data-movement scope to feed AI agents, not just warehouses — Airbyte is an open-source ELT platform syncing APIs, databases and files into warehouses, lakes and now AI applications. It supports self-hosted and Cloud deployments, with a large connector catalog and tooling for data engineers to customize or build new integrations. The MIT/ELv2 licensing keeps the core open while covering long-tail sources.

Python🇺🇸 North AmericaAttention 29
★22,354
juicedata/juicefs

A POSIX Layer That Turns S3 and Redis Into Shared Storage — JuiceFS lets S3-style object storage act like a full POSIX filesystem, with metadata kept in Redis, MySQL, TiKV or similar engines. It connects cloud storage to big data, ML and AI pipelines without code changes, making it useful wherever Hadoop-style HDFS workloads need to move to the cloud.

GoAttention 23
★14,720
benbjohnson/litestream

Disaster Recovery for SQLite, Byte by Byte via the SQLite API — Litestream runs as a background process that incrementally replicates SQLite changes to another file or S3, speaking only through the SQLite API so it never risks corrupting the database. It suits anyone running SQLite in production who wants continuous backup without a separate database server.

GoAttention 39
★14,610
drakkan/sftpgo

An event-driven file transfer server unifying SFTP, FTP/S, WebDAV and cloud storage — SFTPGo lets teams expose S3, Google Cloud Storage, Azure Blob or local filesystems through SFTP, HTTP/S, FTP/S and WebDAV with one configurable server. The free AGPLv3 Community edition covers all protocols and storage backends, while a paid Enterprise edition adds faster cloud I/O and high-availability features for compliance-heavy deployments.

Go🇪🇺 EuropeAttention 29
★12,792
aws/aws-sdk-js-v3

The modular, TypeScript-first rewrite of AWS's JS SDK — AWS SDK for JavaScript v3 splits the monolithic v2 SDK into per-service packages, so you only install what you use. It adds first-class TypeScript types and a redesigned middleware stack, with official migration guides and sample apps for teams moving from v2.

TypeScript🇺🇸 North AmericaAttention 25
★3,682
ModernRelay/omnigraph

A Git-Branching Graph Database for Multi-Agent Context — Omnigraph is a Rust graph database built directly on object storage (via Lance/Arrow/DataFusion), combining typed ontology enforcement, branching workflows, and unified vector/BM25/graph retrieval. It targets long-horizon multi-agent systems that need a shared, durable world model rather than stateless per-session context.

RustAttention 48
★1,969
rootprint/rootprint

Open-source, self-hosted logs and traces with full-text search directly on S3-compatible object storage.

SvelteAttention 12
★443
DahanItamar/DockNest

Self-hosted Docker backup and restore with a visual server map. Move sites between servers safely, encrypted SFTP or S3 storage, verified restores.

JavaScript🌐 Elsewhere
★60
peterretief/yggstore

Pool machines over Yggdrasil for files, mail and sites — A Go toolkit letting people and organisations pool their own boxes over the Yggdrasil mesh, storing encrypted, erasure-coded files, email, websites and messaging without a central server or provider. It adds an S3 gateway for those without hardware and keeps full file history. Status: experimental, crypto unreviewed.

Go
★41