(feat)db: support different sqlite3 libraries (#1907)

doc/org-roam.org

@@ -418,6 +418,51 @@ One can also conveniently insert links via the completion-at-point functions
 Org-roam provides (see [[*Completion][Completion]]).
 
 * Customizing Node Caching
+** How to cache
+
+Org-roam uses a sqlite database to perform caching, but there are multiple Emacs
+libraries that can be used. The default used by Org-roam is ~emacs-sqlite~.
+Below the pros and cons of each package is used:
+
+[[https://github.com/skeeto/emacsql][**emacs-sqlite**]]
+
+The default option used by Org-roam. This library is the most mature and
+well-supported and is imported by default in Org-roam.
+
+One downside of using ~emacs-sqlite~ is that using it requires compilation and
+can cause issues in some environments (especially Windows). If you have issues
+producing the customized binary required by ~emacs-sqlite~, consider using
+~emacs-sqlite3~.
+
+[[https://github.com/cireu/emacsql-sqlite3][**emacs-sqlite3**]]
+
+~emacs-sqlite3~ uses the official sqlite3 binary that can be obtained from your
+system's package manager. This is useful if you have issues producing the
+~sqlite3~ binary required by the other packages. However, it is not recommended
+because it has some compatibility issues with Emacs, but should work for most
+regular cases. See [[https://nullprogram.com/blog/2014/02/06/][Chris Wellon's blog post]] for more information.
+
+To use ~emacsql-sqlite3~, ensure that the package is installed, and set:
+
+#+begin_src emacs-lisp
+  (setq org-roam-database-connector 'sqlite3)
+#+end_src
+
+[[https://github.com/emacscollective/emacsql-libsqlite3/][**emacsql-libsqlite3**]]
+
+~emacs-libsqlite3~ is a relatively young package which uses an Emacs module that
+exposes parts of the SQLite C API to Emacs Lisp, instead of using subprocess as
+~emacsql-sqlite~ does. It is expected to be a more performant drop-in
+replacement for ~emacs-sqlite~.
+
+At the moment it is experimental and does not work well with the SQL query load
+required by Org-roam, but you may still try it by ensuring the package is
+installed and setting:
+
+#+begin_src emacs-lisp
+  (setq org-roam-database-connector 'libsqlite3)
+#+end_src
+
 ** What to cache
 
 By default, all nodes (any headline or file with an ID) are cached by Org-roam.

doc/org-roam.texi

@@ -86,6 +86,7 @@ General Public License for more details.
 * Command Index::
 * Function Index::
 * Variable Index::
+* Bibliography: Bibliography (1). 
 
 @detailmenu
 --- The Detailed Node Listing ---
@@ -109,6 +110,7 @@ Getting Started
 
 Customizing Node Caching
 
+* How to cache::
 * What to cache::
 * When to cache::
 
@@ -695,10 +697,57 @@ Org-roam provides (see @ref{Completion}).
 @chapter Customizing Node Caching
 
 @menu
+* How to cache::
 * What to cache::
 * When to cache::
 @end menu
 
+@node How to cache
+@section How to cache
+
+Org-roam uses a sqlite database to perform caching, but there are multiple Emacs
+libraries that can be used. The default used by Org-roam is @code{emacs-sqlite}.
+Below the pros and cons of each package is used:
+
+@uref{https://github.com/skeeto/emacsql, @strong{@strong{emacs-sqlite}}}
+
+The default option used by Org-roam. This library is the most mature and
+well-supported and is imported by default in Org-roam.
+
+One downside of using @code{emacs-sqlite} is that using it requires compilation and
+can cause issues in some environments (especially Windows). If you have issues
+producing the customized binary required by @code{emacs-sqlite}, consider using
+@code{emacs-sqlite3}.
+
+@uref{https://github.com/cireu/emacsql-sqlite3, @strong{@strong{emacs-sqlite3}}}
+
+@code{emacs-sqlite3} uses the official sqlite3 binary that can be obtained from your
+system's package manager. This is useful if you have issues producing the
+@code{sqlite3} binary required by the other packages. However, it is not recommended
+because it has some compatibility issues with Emacs, but should work for most
+regular cases. See @uref{https://nullprogram.com/blog/2014/02/06/, Chris Wellon's blog post} for more information.
+
+To use @code{emacsql-sqlite3}, ensure that the package is installed, and set:
+
+@lisp
+(setq org-roam-database-connector 'sqlite3)
+@end lisp
+
+@uref{https://github.com/emacscollective/emacsql-libsqlite3/, @strong{@strong{emacsql-libsqlite3}}}
+
+@code{emacs-libsqlite3} is a relatively young package which uses an Emacs module that
+exposes parts of the SQLite C API to Emacs Lisp, instead of using subprocess as
+@code{emacsql-sqlite} does. It is expected to be a more performant drop-in
+replacement for @code{emacs-sqlite}.
+
+At the moment it is experimental and does not work well with the SQL query load
+required by Org-roam, but you may still try it by ensuring the package is
+installed and setting:
+
+@lisp
+(setq org-roam-database-connector 'libsqlite3)
+@end lisp
+
 @node What to cache
 @section What to cache
 
@@ -1006,7 +1055,7 @@ Remove a ref from the node at point.
 
 Since version 9.5, Org has first-class support for citations. Org-roam supports
 the caching of both these in-built citations (of form @code{[cite:@@key]}) and @uref{https://github.com/jkitchin/org-ref, org-ref}
-citations (of form cite:key).
+citations (of form (NO@math{_ITEM}@math{_DATA}:key)).
 
 Org-roam attempts to load both the @code{org-ref} and @code{org-cite} package when
 indexing files, so no further setup from the user is required for citation
@@ -2178,5 +2227,10 @@ When GOTO is non-nil, go the note without creating an entry."
 
 @printindex vr
 
+@node Bibliography (1)
+@chapter Bibliography
+
+NO@math{_ITEM}@math{_DATA}:key
+
 Emacs 28.0.50 (Org mode N/A)
 @bye

org-roam-db.el

@@ -34,6 +34,26 @@
 (defvar org-outline-path-cache)
 
 ;;; Options
+(defcustom org-roam-database-connector 'sqlite
+  "The database connector used by Org-roam.
+This must be set before `org-roam' is loaded. To use an
+alternative connector you must install the respective package
+explicitly. When `sqlite', then use the `emacsql-sqlite' library
+that is being maintained in the same repository as `emacsql'
+itself. When `libsqlite3', then use the `emacsql-libsqlite3'
+library, which itself uses a module provided by the `sqlite3'
+package. This is still experimental. When `sqlite3', then use the
+`emacsql-sqlite3' library, which uses the official `sqlite3' cli
+tool, which is not recommended because it is not suitable to be
+used like this, but has the advantage that you likely don't need
+a compiler. See https://nullprogram.com/blog/2014/02/06/."
+  :package-version '(org-roam . "2.2.0")
+  :group 'forge
+  :type '(choice (const sqlite)
+                 (const libsqlite3)
+                 (const sqlite3)
+                 (symbol :tag "other")))
+
 (defcustom org-roam-db-location (expand-file-name "org-roam.db" user-emacs-directory)
   "The path to file where the Org-roam database is stored.
 
@@ -96,6 +116,22 @@ slow."
   (gethash (expand-file-name org-roam-directory)
            org-roam-db--connection))
 
+(defun org-roam-db--conn-fn ()
+  "Return the function for creating the database connection."
+  (cl-case org-roam-database-connector
+    (sqlite
+     (progn
+       (require 'emacsql-sqlite)
+       #'emacsql-sqlite))
+    (libsqlite3
+     (progn
+       (require 'emacsql-libsqlite3)
+       #'emacsql-libsqlite3))
+    (sqlite3
+     (progn
+       (require 'emacsql-sqlite3)
+       #'emacsql-sqlite3))))
+
 (defun org-roam-db ()
   "Entrypoint to the Org-roam sqlite database.
 Initializes and stores the database, and the database connection.
@@ -104,8 +140,9 @@ Performs a database upgrade when required."
                (emacsql-live-p (org-roam-db--get-connection)))
     (let ((init-db (not (file-exists-p org-roam-db-location))))
       (make-directory (file-name-directory org-roam-db-location) t)
-      (let ((conn (emacsql-sqlite org-roam-db-location)))
-        (set-process-query-on-exit-flag (emacsql-process conn) nil)
+      (let ((conn (funcall (org-roam-db--conn-fn) org-roam-db-location)))
+        (when-let ((process (emacsql-process conn)))
+          (set-process-query-on-exit-flag process nil))
         (puthash (expand-file-name org-roam-directory)
                  conn
                  org-roam-db--connection)

org-roam-utils.el

@@ -68,6 +68,7 @@ Kills the buffer if KEEP-BUF-P is nil, and FILE is not yet visited."
   (declare (indent 2) (debug t))
   `(let* (new-buf
           (auto-mode-alist nil)
+          (find-file-hook nil)
           (buf (or (and (not ,file)
                         (current-buffer)) ;If FILE is nil, use current buffer
                    (find-buffer-visiting ,file) ; If FILE is already visited, find buffer