A comprehensive database engine implementation featuring a multi-layered architecture with storage management, buffer management, record management, and B+ Tree indexing capabilities.
- Project Overview
- System Architecture
- Core Components
- Project Structure
- Getting Started
- Testing
- Learning Outcomes
- Use Cases
- Notes
The Own Database Engine is a complete database management system that demonstrates fundamental database concepts through a layered architecture. This project serves as both an educational tool and a foundation for understanding how modern database systems work internally.
- Implement a complete database engine from scratch
- Demonstrate multi-layered database architecture
- Provide hands-on experience with database internals
- Showcase efficient data structures and algorithms
┌─────────────────────────────────────────────────────────────────────────────┐
│ 🗄️ Database Engine │
│ (User Interface -CLI) │
└─────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ 🌳 B+-Tree Index Layer │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────────────────┐ │
│ │ createBtree │ │ findKey │ │ insertKey │ │
│ │ openBtree │ │ deleteKey │ │ deleteBtree │ │
│ │ closeBtree │ │ openTreeScan │ │ closeTreeScan │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ 📊 Record Management Layer │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────────────────┐ │
│ │ createTable │ │ insertRecord │ │ getRecord │ │
│ │ openTable │ │ deleteRecord │ │ updateRecord │ │
│ │ closeTable │ │ startScan │ │ next │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ 🧠 Buffer Management Layer │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────────────────┐ │
│ │ initBufferPool │ │ pinPage │ │ markDirty │ │
│ │shutdownBufferPool│ │ unpinPage │ │ forcePage │ │
│ │ │ │ │ │ │ │
│ │ Replacement Strategies: FIFO, LRU, Clock, LFU, LRU-K │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ 💾 Storage Management Layer │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────────────────┐ │
│ │ createPageFile │ │ readBlock │ │ writeBlock │ │
│ │ openPageFile │ │ readNextBlock │ │ writeCurrentBlock │ │
│ │ closePageFile │ │ readCurrentBlock│ │ appendEmptyBlock │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ 📁 File System │
│ (Page Files, Index Files) │
└─────────────────────────────────────────────────────────────────────────────┘
- Purpose: Provides efficient indexing for fast data retrieval and manipulation
- Features:
- Create, open, close, and delete B+-tree indexes
- Insert, delete, and search operations
- Tree scanning capabilities
- Support for Integer data types
- Location:
B+Tree Manger/
- Purpose: Manages table operations, record storage, and schema handling
- Features:
- Table creation, opening, and deletion
- CRUD operations (Create, Read, Update, Delete)
- Conditional scanning with search conditions
- Schema management and record serialization
- Location:
Record Manager/
- Purpose: Implements memory management with multiple page replacement strategies
- Features:
- Multiple replacement strategies (FIFO, LRU, Clock, LFU, LRU-K)
- Page pinning and unpinning
- Dirty page management
- Buffer pool statistics and monitoring
- Location:
Buffer Manager/
- Purpose: Handles low-level file I/O operations and page management
- Features:
- Page file creation, opening, and closing
- Block reading and writing operations
- Sequential and random page access
- File capacity management
- Location:
Storage Manager/
Own-Database-Engine/
├── 🌳 B+Tree Manger/ # B+-Tree indexing implementation
├── 🧠 Buffer Manager/ # Buffer pool management
├── 📊 Record Manager/ # Record and table management
├── 💾 Storage Manager/ # File I/O and page management
└── 📖 README.md
- C compiler (GCC recommended)
- Make utility
- Basic understanding of database concepts
Each component can be built independently:
# B+-Tree Manager
cd "B+Tree Manger"
make -f makefile.mk
# Buffer Manager
cd "../Buffer Manager"
make -f makefile.mk
# Record Manager
cd "../Record Manager"
make -f makefile.mk
# Storage Manager
cd "../Storage Manager"
make -f makefile.mkEach component includes its own test suite:
# B+-Tree Manager
cd "B+Tree Manger"
make -f makefile.mk
./btree_mgr
# Buffer Manager (FIFO)
cd "../Buffer Manager"
make -f make_file.mk
./buffermgr
# Buffer Manager (LRU)
cd "../Buffer Manager"
make -f makefile.mk
./buffer_mgr
# Record Manager
cd "../Record Manager"
make -f makefile.mk
./record_mgr
# Storage Manager (Basic)
cd "../Storage Manager"
make -f makefile.mk
./assign1
# Storage Manager (Advanced)
cd "../Storage Manager"
make -f makefile2.mk
./assign2Each component includes comprehensive test suites:
- Storage Manager: Basic and advanced file I/O operations
- Buffer Manager: FIFO and LRU replacement strategies
- Record Manager: Table operations and record handling
- B+-Tree Manager: Index operations and tree management
This project demonstrates:
- Database Architecture: Multi-layered design principles
- Memory Management: Efficient buffer pool strategies
- Data Structures: B+-tree implementation and optimization
- File I/O: Low-level storage management
- Error Handling: Robust error management systems
- Performance Optimization: Multiple replacement strategies
- Educational: Learning database internals
- Research: Database algorithm experimentation
- Development: Foundation for custom database systems
- Prototyping: Testing database design concepts
- Each component is designed to work independently
- Components can be integrated for full database functionality
- Error handling is consistent across all layers
- Performance optimization is a key focus area