Waiting for input...
Star SPIKE on GitHub

ADR-0014: Maintaining SQLite as SPIKE’s Primary Storage Backend


  • Status: accepted
  • Date: 2024-11-15
  • Tags: Persistence, Storage, SQLite, File-System

Context

SPIKE currently uses SQLite as its backing store for secret management. There is a proposal to implement a file-system-based backing store as an alternative storage solution. SPIKE uses SPIFFE as its identity control plane for authentication and authorization.

The key considerations are:

  • SQLite is already implemented and proven to work well for SPIKE’s needs
  • SQLite itself is fundamentally a file-system-based database
  • A new file-system backing store would need to implement similar functionality
  • Both solutions ultimately interact with the filesystem at their core

Decision

We will maintain SQLite as the primary storage backend and not implement a separate file-system-based storage solution.

Rationale

Technical Advantages of SQLite

  1. File-System Foundation: SQLite already operates directly on the file system, using efficient file I/O operations. As per SQLite’s documentation, it competes directly with fs.open() for performance.

  2. ACID Compliance: SQLite provides built-in:

    • Atomicity for operations
    • Consistency in data storage
    • Isolation for concurrent operations
    • Durability of stored data

  3. Proven Security: SQLite has undergone extensive security auditing and has a well-understood security model.

Implementation Considerations

  1. Redundant Development: Creating a separate file-system store would:

    • Duplicate existing functionality
    • Require implementing concurrent access controls
    • Need additional security auditing
    • Require new testing infrastructure

  2. Maintenance Overhead: Supporting two storage backends would:

    • Increase maintenance complexity
    • Require maintaining two sets of documentation
    • Complicate troubleshooting
    • Potentially create inconsistencies in behavior

Performance

  • SQLite is highly optimized for the types of operations SPIKE performs
  • The overhead of SQLite compared to direct file system operations is negligible for SPIKE’s use case
  • SQLite’s page cache provides performance benefits that would need to be re-implemented in a file-system solution

Consequences

Positive

  1. Reduced development effort and maintenance burden
  2. Continued use of a well-tested, secure storage solution
  3. Simplified codebase and deployment
  4. Consistent behavior across deployments

Negative

  1. Less flexibility in storage formats
  2. Continued dependency on SQLite
  3. Potential perception of over-engineering for simple storage needs

Mitigations

  1. Document SQLite’s role as a file-system interface
  2. Maintain clear interface boundaries for potential future storage implementations
  3. Monitor performance and storage requirements to validate continued suitability