Operating System Support for Database Management

Stonebraker

One line summary:

Examine various services the OS provides but DBMS cannot take advantage of.

Buffer Pool Management:

   * The overhead of fetching a block from the buffer pool is too high. So
     the DBMS usually manages its own buffer pool in user space. A potential
     solution is mapping the file system into virtual memory.
   * LRU replacement strategy performs only marginally in a database
     environment. DBMS knows it access patterns, so should be able to give
     "advice" to OS on replacement policy.
   * Most DBMS knows the blocks to be accessed next, so instead of OS doing
     sequential (in the logical file order) prefetching, it should be able
     to give "advice" again.
   * The way some DBMS do crash recovery requires data to be flushed to disk
     before the commit flag. But DBMS has no control on the order of
     flushing pages.

File System:

   * Lack of clustering: Want segments instead of blocks/pages.
   * OS maintains trees for file blocks and directories and DBMS also for
     indices. Want to unify them to reduce the overhead in managing trees.

Scheduling, Process Management and Interprocess Communication

   * How to implement a multiuser database?
        o Process per user: expensive to context switch. Also to deschedule
          a process holding a lock for a critical section can cause other
          process/user to queue up for the locked resource.
        o Server process for all users: The server needs to implement
          scheduling and multitasking, duplicating OS facilities. Some
          solutions: FCFS; pool of server process; pool of server process
          along with disk processes.
        o Message system: Too much overhead here. Are messages much cheaper
          today?
   * So there are tradeoffs involved here.

Consistency control:

   * Want finer granularity locking than what OS provides. (instead of
     file-level locking, want page-level locking)
   * Buffer management and consistency work with each other. If one is in
     user space and the other in OS, there is duplication in effort.

Paged VM:

   * Bind files into user's paged virtual address space. So a user never
     needs to do explicit reads or writes; can depend on the OS facilities
     to move files blocks into and out of main memory.
   * Problems with large files: page table might need to be paged. (Memory
     is much cheaper today, but also is DB)
   * Bind chunks of a file into a user's address space. Bookkeeping is a
     pain.

Comments:

   * How much of these complaints are invalid based on today's technology?
   * How to implement DBMS below FS? vice versa?
   * Exokernel? SPIN?