Access Your Password-Manager from Emacs on Linux Using KeePassXC

Emacs is great. Nothing to debate here. As you probably know, you can use Emacs for nearly anything, as you would expect from any good operating system. But when using Emacs for everything, you are going to run into a situation where you will need to use passwords in Emacs. And you don't want to put them in a plain text file, do you?

I wanted to try GPTel today, a script that allows to use ChatGPT directly from Emacs. GPTel requires an API key to work and when using that API key my credit card will be charged. So, I thought, let's not put API keys into my dotfiles and push them to GitHub, okay? — But how to do it properly?

Emacs and it's Authentication Manager

It turns out, that Emacs has a library build-in, called auth-source.el. Auth-source is the default way to handle passwords and such. This library can pull authentication info from multiple places, that can be configured through the auth-sources configuration variable. Its default value is ("~/.authinfo" "~/.authinfo.gpg" "~/.netrc"), hence some files on my disk which don't even exist.

Using this, you can just pull in authentication info through Auth-sources by searching for a key-value pair in one of your auth-sources, e.g. like this:

(auth-source-search :host "https://api.openai.com")

That is well and good if you decide to use one of these places to store all your secrets, you can even store them encrypted (what you would expect, hopefully). But if you have your secrets already in a password manager, this won't cut it.

Other Auth-sources

Luckily, auth-sources can use other sources, even custom build ones and pull secrets from there. That's great. What is the best approach for my Linux machine though?

Many Linux distributions make use of the Secret Service API. No joke, that's the name 🕶. This API is used to access secrets through a standardized interface using DBus. Any program can request secrets from a secret provider (i.e. a password manager) or request to store a secret.

Many password managers support Secret Service. KeePassXC, the password manager of my choice, does, too. First, though, you have to enable Secret Service in the settings and add your database to the "exposed database groups".

And the great thing: Emacs also supports Secret Service through the secrets.el library. In theory, all you need to do to get your secret is this:

(require 'secrets nil t)
(secrets-open-session)
(secrets-get-secret "MyPws" "ChatGPT API Key")

With the right settings in KeePassXC, this would actually work, but notice that I checked the "Confirm when passwords are retrieved by clients" checkbox in my settings. A, as I tend to think, very reasonable setting.

Just, the secrets library does not agree. When trying to get a password, using secrets-get-secret you will receive a IsLocked error 😭. It could have all been so nice, but I guess it was not meant to be. Well, then, I will keep storing my passwords in plaintext in my Emacs configuration. See you next time...

Naaaah. Of course not.

Fixing an Emacs Bug

First I filed an issue with KeePassXC, but one of the maintainers pointed out to me, that the issue most likely lies in an incomplete or wrong implementation of the Secret Service API. So I dug into how secrets.el works. And what is there to say, he was right.

;;; secrets.el

(defun secrets-item-path (collection item)
  "Return the object path of item labeled ITEM in COLLECTION.
If there are several items labeled ITEM, it is undefined which
one is returned.  If there is no such item, return nil.

ITEM can also be an object path, which is returned if contained in COLLECTION."
  (let ((collection-path (secrets-unlock-collection collection)))
    (or (and (member item (secrets-get-items collection-path)) item)
        (catch 'item-found
          (dolist (item-path (secrets-get-items collection-path))
	    (when (string-equal
                   item (secrets-get-item-property item-path "Label"))
	         (throw 'item-found item-path)))))))

(defun secrets-get-secret (collection item)
  "Return the secret of item labeled ITEM in COLLECTION.
If there are several items labeled ITEM, it is undefined which
one is returned.  If there is no such item, return nil.

ITEM can also be an object path, which is used if contained in COLLECTION."
  (let ((item-path (secrets-item-path collection item)))
    (unless (secrets-empty-path item-path)
      (dbus-byte-array-to-string
       (nth 2
	(dbus-call-method
	 :session secrets-service item-path secrets-interface-item
	 "GetSecret" :object-path secrets-session-path))))))

What secrets.el does, when retrieving a secret is this:

1. It asks secrets-item-path what the path is, it should be asking for.
2. secrets-item-path requests to unlock the collection (i.e. database) and gets all the metadata. It finds the path to the entry we are searching for and returns it.
3. secrets-get-secret tries to retrieve the secret. And fails.

secrets-get-secret assumes, that this is enough to retrieve the secret with only the database unlocked, which actually works in many cases. But thanks to that setting I enabled in KeePassXC, KeePassXC wants to ask me first, if Emacs is allowed to access that particular secret. So it does not just hand out the password and calls it a day but returns IsLocked, saying that that particular password is still locked.

Thanks to the Emacs madness of having everything in global scope, we can fix this issue. What we need to do is to request KeePassXC to unlock that particular item we want for us. Let's see how we can do that:

  (defun secrets-unlock-item (collection item)
    "Unlock item labeled ITEM from collection labeled COLLECTION.
  If successful, return the object path of the item."
    (let ((item-path (secrets-item-path collection item)))
      (unless (secrets-empty-path item-path)
        (secrets-prompt
         (cadr
          (dbus-call-method
           :session secrets-service secrets-path secrets-interface-service
           "Unlock" `(:array :object-path ,item-path)))))
      item-path))

This new secrets-unlock-item function

1. retrieves the item path from secrets-item-path as before. secrets-item-path unlocks the database as before.
2. With the correct item path secrets-unlock-item requests to unlock the item.
3. secrets-unlock-item returns the item path.

Now, we need to update the secrets-get-secret function to use our new secrets-unlock-item function, and we are good to go.

  (defun secrets-get-secret (collection item)
    "Return the secret of item labeled ITEM in COLLECTION.
    If there are several items labeled ITEM, it is undefined which
    one is returned.  If there is no such item, return nil.

    ITEM can also be an object path, which is used if contained in COLLECTION."
-    (let ((item-path (secrets-item-path collection item)))
+    (let ((item-path (secrets-unlock-item collection item)))
      (unless (secrets-empty-path item-path)
        (dbus-byte-array-to-string
         (nth 2
              (dbus-call-method
               :session secrets-service item-path secrets-interface-item
               "GetSecret" :object-path secrets-session-path))))))

I opened a ticket on the Emacs bug tracker to incorporate this change but until then, you can just add secrets-unlock-item and the modified secrets-get-secret to your configuration and be good to go.

Putting it together

Finally, how can we use GPTel with my API key loaded from my password manager?

Step 0: Load secrets.el:

;; Actually, it makes a lot of sense to defer loading secrets. For
;; that see at the end of the post.
(require 'secrets nil t)

Step 1: We need to add secrets.el as a source to auth-sources using M-x customize-variable. We will add secrets:MyPws to the top of the list, where MyPws is the name of the database as shown in KeePassXC.

Step 2: We need to update the gptel-api-key variable to be a function that retrieves the API key for us:

(lambda nil
  (plist-get
   (nth 0
        (auth-source-search :Title "ChatGPT"))
    ;; In my case, I stored the API key not in the password field but
    ;; in an extra field called "API Key". If you need to access the
    ;; password field, use :secret instead
   :API\ Key))

Step 3: ...

Step 4: Profit.

Proof that it works: I asked ChatGPT to rewrite that last step for me. This is what it came up with:

Step 4: Achieve Profitability.

— ChatGPT

Couldn't have said it better. Golden. 👑

The code

I threw use-package at secrets.el to defer loading and thus prevent Emacs from taking forever to start up in case no secret service provider is found via DBus.

  ;; Load and patch secrets
  (use-package secrets
    :commands (secrets-search-items
               secrets-get-secret
               secrets-get-attributes)
    :config
    ;; Adds a patch to fix behavior with KeepassXC
    (defun secrets-unlock-item (collection item)
      "Unlock item labeled ITEM from collection labeled COLLECTION.
    If successful, return the object path of the item."
      (let ((item-path (secrets-item-path collection item)))
        (unless (secrets-empty-path item-path)
          (secrets-prompt
           (cadr
            (dbus-call-method
             :session secrets-service secrets-path secrets-interface-service
             "Unlock" `(:array :object-path ,item-path)))))
        item-path))

    ;; Adds a patch to fix behavior with KeepassXC
    (defun secrets-get-secret (collection item)
      "Return the secret of item labeled ITEM in COLLECTION.
    If there are several items labeled ITEM, it is undefined which
    one is returned.  If there is no such item, return nil.

    ITEM can also be an object path, which is used if contained in COLLECTION."
      (let ((item-path (secrets-unlock-item collection item)))
        (unless (secrets-empty-path item-path)
          (dbus-byte-array-to-string
           (nth 2
                (dbus-call-method
                 :session secrets-service item-path secrets-interface-item
                 "GetSecret" :object-path secrets-session-path)))))))
                 
  ;; Use auth-source.el with secret.el to load an API key
  (use-package gptel
    :ensure t
    :config
    (setq gptel-api-key #'(lambda nil
                          (plist-get
                           (nth 0
                                (auth-source-search :Title "ChatGPT"))
                           :API\ Key))))

Secret.el functions to know

Lastly, I want wrap this up with a snippet of the most important secret.el functions and how we can use them.

(require 'secrets nil t)

;; Get all attributes of the ChatGPT entry
(secrets-get-attributes "MyPws" "ChatGPT")

;; Get just the "API Key" attribute from the ChatGPT entry
(secrets-get-attribute "MyPws" "ChatGPT" :API\ Key)

;; Get the password from the "ChatGPT" entry
(secrets-get-secret  "MyPws" "ChatGPT")

;; Show all secrets
(secrets-show-secrets)

;; Unlock the collection (should be done automatically)
(secrets-unlock-collection "MyPws")

;; Open/close the session (should be done automatically)
(secrets-open-session)
(secrets-close-session)

We can use secrets-get-attribute e.g. when using restclient mode like this:

:openai-api-key := (secrets-get-attribute "MyPws" "ChatGPT" :API\ Key)

### Get ChatGPT completion
POST https://api.openai.com/v1/chat/completions
Content-Type: application/json
Authorization: Bearer :openai-api-key

{
    "model": "gpt-3.5-turbo",
    "messages": [{
        "role": "user",
        "content": 
            "Give me a short description of Restclient mode in Emacs."
    }]
}

"Restclient mode in Emacs is a major mode used for testing RESTful web services. It provides a simple and convenient interface for sending HTTP requests along with headers, query parameters, request bodies and authentication credentials. The response from the server can be viewed in a separate buffer and parsed using various JSON or XML tools. Restclient mode is a helpful tool for developers to test their web services without the need for a dedicated REST client."

— ChatGPT