How to Encrypt Field Data in Drupal 8

Getting started: If this is the first module you’ve used on the site that requires the Encrypt module, the first thing you’ll need to do is install and configure that module and its dependencies.

Take a look at this post for How to Setup Encrypt Module for Drupal 8, then come back here to continue setting up the Field Encrypt module. I’ll wait.

Welcome back! Now let’s setup the Field Encrypt module!

Modules

As with many Drupal tasks, to achieve this goal we’re going to need at least 4 modules: Key, Encrypt, Real AES, and Field Encrypt. Each of these modules performs one specific function that the others utilize. Let’s look at each one and how it fits into the others.

Field Encryption

https://www.drupal.org/project/field_encrypt

Adds options to encrypt field values. The goal of this module is to create a method for encrypting field values when stored in the database.

The Field Encryption module provides a UI for managing the encryption settings for fields within Drupal. After enabling the module, you can edit the Storage Settings on a field and configure the field to encrypt its data within the database, along with some other settings.

Field Encrypt: Automatically encrypt field data at rest

Now that all the pieces are in place, we can start configuring our fields to encrypt their data within the database.

There are two general areas of configuration with the Field Encrypt module. First is the site-wide default settings. To view these settings navigate to Configuration > System > Field Encrypt Settings (/admin/config/system/field-encrypt).

Here you can configure which properties of a field type are checked by default when enabling field encryption. For example, if you have the Address field installed on your site and know that every time you choose to encrypt an Address field you want the the address_line1 and postal_code properties to be encrypted, you can set those here as the default.

Basically, this settings page is just to give you the administrator some common encryption defaults between field types. Changing the settings here does not affect any of your existing fields. You don’t necessarily need to modify these settings at all.

Next up, let’s configure a field to make it encrypt its data!

For the sake of example, let’s add a new field to the Basic Page content type and setup that field to be encrypted. This will be a simple text field. After we configure the field and create some data, we’ll look at the database to see how the data is stored.

  1. Navigate to Structure > Content Types > Basic Page > Manage Fields (/admin/structure/types/manage/page/fields) and click “Add Field”
  2. Field Type: choose “Text (plain)”
  3. Field Name: “Super Cool Encrypted Field”, or another name of your choosing.
  4. Click “Save and Continue”
  5. On the next page, beneath the normal Field Storage options, you should see a new checkbox labeled “Encrypt Field”. Check that box and the form will expand with some options.
  6. Properties: check “Text value” – Here we’re choosing which parts of the field data will be encrypted. Since this is a plain text field, there is only one property available to encrypt.
  7. Encryption Profile: choose “Field Encryption AES Profile” – or whatever you named the encryption profile in the previous setup.
  8. Uncacheable: Check this. This will make sure your unencrypted data will not be exposed in the cache, but will have a negative impact on your performance. Whether or not you want encrypted data to be cacheable in your actual usage of the module is up to you, but there are trade-offs in both cases. “Uncacheable” is the more secure option, but has a performance cost.
  9. Click “Save Field Settings”

Screenshot

screenshot of filed encrypt settings on a plain text field

And we’re done! You should now have a field that encrypts its data on saving to the database, and decrypts the data when shown to a site user. ?

Testing & Exploring the data

Let’s test this out and see what is happening behind the scenes to our data.

Create a new Basic Page and provide some text to our newly encrypted field. Everything should appear normal to you. The field data is shown decrypted when viewing or editing the node.

Screenshot: All is normal

screenshot of node form and display

But if we were to look at our new field in the database we will not find the data. Instead, the value of the field in the database is literally “[ENCRYPTED]”.

screenshot of field data in database
Note: I’m using a content type called “Test Dev” in this example, as opposed to a Basic Page.

This is because encrypted data for fields is now stored in a few new database tables provided by the field_encrypt module.

screenshot of encrypted data in database
Note: There are 2 rows shown here because I have revisions enabled for the content type.

There it is, my encrypted data.

Now to complete the circle on testing this and understanding how all these pieces fit together, I should be able to copy the encrypted data from the database, and Test Decrypt it with my Encryption Profile.

Screenshot of decrypting data taken from the database

screenshot of decrypting the data copied from the database

And viola! Works like a charm.

So there we have it, we are now encrypting field data at rest using a few modules that provide simple administrative UIs and are designed to work together.

Additional Notes and Gotcha’s

As mentioned in a previous section, when you create an encryption profile with the Encrypt module, that profile becomes available to a global encryption service within your code. Though you may never need to manage your own encryption/decryption in code, let’s take a quick look at that for reference.

Programmatically Encrypt & Decrypt Data

Assuming you have created an encryption profile named “Field Encrypt AES” (with the machine name: field_encrypt_aes), the following code is an example of using that profile programmatically.

use Drupal\encrypt\Entity\EncryptionProfile;

$string = "I bet this is going to work great!";
$encryption_profile = EncryptionProfile::load('field_encrypt_aes');

$encrypted = Drupal::service('encryption')->encrypt($string, $encryption_profile);
$decrypted = Drupal::service('encryption')->decrypt($encrypted, $encryption_profile);

dsm($encrypted);
dsm($decrypted);

Note: This example is both encrypting and decrypting right after the other. In the real world, you would do these things separately depending on the events taking place in the system.

Screenshot: Seeing is believing

screenshot of programmatic encryption using devel PHP

Note how all you need to do is load the encryption profile by machine name, and you’re ready to en/decrypt! Easy enough!

Key: Encryption keys as a service with Lockr

As mentioned in the best practices section for the Key module, the ideal secure setup is storing your key on a server that is complete separate from your Drupal application. There are likely a few good ways to do this, but of note is the service provided by Lockr.

Lockr offers keys as a service. Meaning they will generate, store, and provide API access to your encryption keys. This not only solves the problem of storing your keys securely, but it offers an additional benefit to easily scale your encryption configuration.

Consider adding encryption to an application that is load-balanced across multiple servers. If you’re encrypting data within your application, then each server will need access to the keys used for that encryption.

How to use Lockr

  1. Visit https://www.lockr.io and create a new account.
  2. Download the Lockr module and its dependencies to your Drupal instance – https://www.drupal.org/project/lockr and install it.
    • Installing with composer will automatically download the dependencies you need.
      composer require drupal/lockr
    • If you download the module another way, you will need to also download the lockr-client library to Drupal’s vendor directory – https://github.com/lockr/lockr-client
  3. After installing Lockr, visit the module’s configuration page by navigating to Configuration > System > Lockr (/admin/config/system/lockr)
  4. Here you will be guided through generating a certificate on your machine, and providing your Lockr username and password.
  5. Once the Lockr module configuration is complete visit the Key module’s configuration page at Configuration > System > Keys and click “Add Key”
  6. Key Type: select “Lockr Encryption”
  7. Key Size: select “256”
  8. Key Provider: select “Lockr”
  9. Save your new Key

Now the only thing left to do is to modify your existing Encryption Profile (or create a new one) that uses this new Lockr Key. The rest of the Field Encryption setup is the same as above.

Screenshot of Lockr Key Configuration

screenshot of key using Lockr service

 

Gotcha #1: Encrypting fields with existing data

It’s worth pointing out that at the time of this writing the field_encrypt module has a bug related to its use of cron to encrypt existing data. In testing this out I had cron hard-crash a few times and re-verified my setup to make sure I had everything configured correctly.

Luckily there is an issue for this in the queue with a working patch: https://www.drupal.org/node/2900641

Gotcha #2: Adding encryption to fields on custom entities

Also while experimenting with these modules I ran into a very confusing issue dealing with custom entities. If you add encryption to an existing field on a custom entity, you must clear your site cache before it will work correctly.

This really threw me for a loop when working with the modules, as all of my data appeared as “[ENCRYPTED]” on both entity display and the entity form. After determining there was a bug of some sort at play, I submitted an issue to the Field Encryption module: https://www.drupal.org/node/2918252

Conclusion

That’s all I have for now on data encryption at rest in Drupal 8, but I fully expect to be digging further into the modules and learning more ways to secure my site data.

If you’ve made it this far in the post let me know what you think. If you see any ways this post could be improved or have any questions about these modules, let me know and I’ll figure it out!

Update: Now in full-color video!

10 Thoughts

Discussion

Brandt Kurowski
January 31, 2018

Great post, Jonathan. I already had some idea of what most of the pieces looked like, but reading your writeup of all the details and steps made explicit saved me a bunch of time. Thanks!

Stefan Weber
March 22, 2018

Hi Jonathan,
really helpful post, specially with GDPR coming up here in the EU.
I would like add two modules to the list that integrate nicely with this setup:
1) DataBase Email Encryption (https://www.drupal.org/project/dbee) for encrypting the users email addresses (currently, use dev version, since it uses real_aes instead of aes)
2) Webform Encrypt (https://www.drupal.org/project/webform_encrypt) for encrypting data from webform submissions.
These are the most common use cases when it comes to user data protection.

Gerald
June 3, 2018

Love the post Jonathan, it helped me a lot.
I would like to run a query to fetch the encrypted_value__value from the table encrypted_field_data, but the method Drupal::service(‘encryption’)->encrypt() gives me a different encrypted value like “def50200e50b…..” and the encrypted_value__value in the database is like “ZGVmNTAyMDA3NTY3NTY….”.
Is it possible to db select these by the incrypted value?

Jonathan Daggerhart
June 3, 2018

Hi Gerald,

Glad this post was helpful!

I believe the values stored in encrypted_field_data.encrypted_value__value table are both encrypted and base64_encode()ed. You’ll want to do the same with the value you’re trying to select.

Something along the lines of:

$string = "The value we're searching for.";
$encrypted = Drupal::service('encryption')->encrypt($string, $encryption_profile);
$encoded = base64_encode($encryped);

… and then query for the $encoded value.

Hope this helps, and let me know how this turns out!

Gerald
June 4, 2018

Hi Jonathan,

I would like to thank you for your fast response. I tried your solution and this is the result :) see the code underneath. i hope that im clear enough. Hope that you can see whats im doing wrong.

========
$encrypted = \Drupal::service(‘encryption’)->encrypt(‘Candy’, $encryptProfile);
echo $encrypted;
/*
This is the output
*/
“def50200e9e1ef124d870725003559565972e39e14e8d3e351ff535dbe84bc6f62e49aff7ed85b988f993972ab7352bc5140855afdd4dd35792cbe4825bab4d2abfb0149d9460656b0029c296273585729b3b0d4dde3774dae”

$decrypted = \Drupal::service(‘encryption’)->decrypt(‘def50200e9e1ef124d870725003559565972e39e14e8d3e351ff535dbe84bc6f62e49aff7ed85b988f993972ab7352bc5140855afdd4dd35792cbe4825bab4d2abfb0149d9460656b0029c296273585729b3b0d4dde3774dae’, $encryptProfile);
echo $decrypted;
/*
This is the output
*/
“Candy”

======== AFTER DOING ======

$encrypted = \Drupal::service(‘encryption’)->encrypt(‘Candy’, $encryptProfile);
$encoded = base64_encode($encryped);

echo $encoded;
/*
This is the output
*/
“ZGVmNTAyMDA4OWRiNTczMTI2ODYxOTA4NGRkNGYxYjg4OWVmYmMzZGU3MWU4ZjA3MjJjMmVmOThkYzc3NmI5ZmZmYjIxOGE4MTNkZjU1Y2NiODliNGI0ODlmOWU1NTBjZWQxMzAwNTQxYjM0NGRhNjQ0NTIzMWI3OWViOTA3M2E5OWJkYjQ1ZWMxOTA5YjZmZDk0ZDc5YzQ2OGFmMzRmNmQxY2UwNjExMGIzZTY4NDhjZA==”

$decrypted = \Drupal::service(‘encryption’)->decrypt(base64_decode(‘ZGVmNTAyMDA4OWRiNTczMTI2ODYxOTA4NGRkNGYxYjg4OWVmYmMzZGU3MWU4ZjA3MjJjMmVmOThkYzc3NmI5ZmZmYjIxOGE4MTNkZjU1Y2NiODliNGI0ODlmOWU1NTBjZWQxMzAwNTQxYjM0NGRhNjQ0NTIzMWI3OWViOTA3M2E5OWJkYjQ1ZWMxOTA5YjZmZDk0ZDc5YzQ2OGFmMzRmNmQxY2UwNjExMGIzZTY4NDhjZA==’), $encryptProfile);
echo $decrypted;
/*
This is the output
*/
“Candy”

======== AFTER GRABBING THE DB DATA ======

$decrypted = \Drupal::service(‘encryption’)->decrypt(‘ZGVmNTAyMDBmOWRhZTcxMTIzZjRkMDkzMmZiMTBhM2JlNTUzNjQ3MTlhOGYwMmUwNmVlMWQ2OGY5NjUyZTIwOGY0MDIyNTRjN2JkZjEwYTczY2U4MTY3NGM0MDVjNzA5MTAxMzA4NGU5ZDM2OGMwZGZhMmJlZDU3ZmRlMmY1YTA4ZTIxYzE0NWNhZjgwZmU1YzY1ODA0OTRmNWYxOTA3NzVlMzQ4YjRhZDM2NDM2MmVmZg==’, $encryptProfile);
var_dump($decrypted);
echo $decrypted;
/*
This is the output
*/
“Candy”

======== BUT HERE IT COMES : ) ======

The encrypted string from the DB does not match mine encrypted value

// Mine
ZGVmNTAyMDA4OWRiNTczMTI2ODYxOTA4NGRkNGYxYjg4OWVmYmMzZGU3MWU4ZjA3MjJjMmVmOThkYzc3NmI5ZmZmYjIxOGE4MTNkZjU1Y2NiODliNGI0ODlmOWU1NTBjZWQxMzAwNTQxYjM0NGRhNjQ0NTIzMWI3OWViOTA3M2E5OWJkYjQ1ZWMxOTA5YjZmZDk0ZDc5YzQ2OGFmMzRmNmQxY2UwNjExMGIzZTY4NDhjZA==

//DB
ZGVmNTAyMDBmOWRhZTcxMTIzZjRkMDkzMmZiMTBhM2JlNTUzNjQ3MTlhOGYwMmUwNmVlMWQ2OGY5NjUyZTIwOGY0MDIyNTRjN2JkZjEwYTczY2U4MTY3NGM0MDVjNzA5MTAxMzA4NGU5ZDM2OGMwZGZhMmJlZDU3ZmRlMmY1YTA4ZTIxYzE0NWNhZjgwZmU1YzY1ODA0OTRmNWYxOTA3NzVlMzQ4YjRhZDM2NDM2MmVmZg==

Jonathan Daggerhart
June 4, 2018

I went a bit deep into testing this and you’re absolutely right. Each time you encrypt the same value you’ll get a different a result because of how the data is being encrypted. I took a look at how REAL AES module is doing this and it involves both salting and noncing the value before hasing it… at which point I lost the thread of what was going on.

Really sorry for sending you down this rabbit hole. I’ll ask around and see if I can find any suggestions for how you might be able to query for a specific value in the database, but after looking at this I’m a bit doubtful.

Gerald
June 5, 2018

Jonathan, thank you! Well your post will help lots of lost souls like it helped me. But I think the encrypted values are going to be random values, so it will be impossible to query the data from the database. But for now i will encrypt only values that are not searchable. Thank u for looking and hope we will find a solution, if i find something i will post it here ;).

Karu
March 19, 2019

Hey Jonathan,

Thanks for the demo, made it super easy to setup.

Now it’s all working as expected, we’re trying to send the encrypted data to another VB.net system to automatically decrypt the data on the fly using our key plus the hashed data and push it into another API. The issue we’re coming up against is, the other system doesn’t recognise the key we created. I’ve tried a few different methods to create this but thought I’d check with you if you’ve had similar issues with this in the past yourself?

Essentially once a key is created, and the data has been encrypted, you should be able to reuse that key on any standard aes encryption/decryption site to decrypt the text with the key (as a string). No matter what we do, the key isn’t being recognised as valid so thought I’d ask?

Jonathan Daggerhart
April 16, 2020

That’s a great article for a really interesting idea. Thanks for sharing!

Philipp
October 4, 2021

Thanks for good description!
I testet it in Drupal9 in custom description.

This works -> EncryptionProfile::load(‘ ‘);
But this doen’t work: Drupal::service(‘encryption’)->encrypt($value, $encryption_profile);
The websites doesnt work.
The Error log says:
Error: Class ‘Drupal\\custom_module\\Plugin\\views\\filter\\Drupal’ not found in /var/www/html/…./dp9/modules/custom_module/src/Plugin/views/filter/CustomAzFilter.php on line 190
[…]

Any ideas? Or is there a Specification in Drupal9 ? Sorry, i’m a Beginner in custom development.
Thanks for any help!

Jonathan Daggerhart
October 5, 2021

You need to add a “\” before the word Drupal. Like:

\Drupal::service('encryption')->encrypt($value, $encryption_profile);
Philipp
October 6, 2021

Great, Thanks Jonathan! The backslash was the problem in \Drupal::service() – maybe special/new in Drupal9

To decrypt works, but encrypt doesn’t with saved data. The data i encrypted and saved in database cannot be decrypted. Is there a timestamp or something else why it does not work?
Domain, encryption_profile is the same.

Patrick
September 19, 2022

Thanks for this post; it’s very informative. The article shows a screenshot for a “Test Decrypt” settings screen– where is this? I can’t find it in the UI for Field Encrypt or Encrypt.

Leave a Reply

Your email address will not be published. Required fields are marked *