End to end encryption in Go with Pusher Channels

Introduction
Understanding encrypted channels
Implementing encrypted channels
Prerequisites
Building the server
Building the client
Conclusion

Introduction

Privacy is a hot topic this days. Who has access to what and who can read my conversation with a friend. Pusher Channels offers three kinds of channels:

Public
Private
Encrypted

To get started with Pusher Channels, create a free sandbox Pusher account or sign in.Basically, all three perform the same functions - flexible pub/sub messaging and tons of others. But there are few differences between them. Public channels do not require client-server authentication in order to subscribe to events. Private channels take it a step further by requiring client-server authentication. Encrypted channels build on top of private channels by introducing security in the form of encrypted data.

Kindly take a look at the images above and spot the difference. Seen any yet ? In the first image which shows the Debug console for a public channel, you can see the data being sent to Pusher Channels contains some fields - title, content and createdAt. Now take a look at the second image, you will notice those fields are no longer present but instead you have a bunch of non-human readable content your application obviously didn’t create. The field called ciphertext is what the data you sent to Pusher Channels was converted to. The word ciphertext outside this discourse refers to encrypted and/or garbled data.

Understanding encrypted channels

As depicted above, an advantage of an encrypted channel is the ability to send messages only the server SDK and any of your connected clients can read. No one else - including Pusher - will be able to read the messages.

NOTE: A client has to go through the authentication process too.

Pusher Channels uses one of the current top encryption algorithms available and that is Secretbox. On the server side, the application author is meant to provide an encryption key to be used for the data encryption. This encryption key never gets to Pusher servers, which is why you are the only one that can read messages in an encrypted channel.

But a question. If the encryption key never gets to Pusher servers, how is a connected client able to subscribe to an event in an encrypted channel and read/decrypt the message ? The answer resides in the authentication process. During authentication, a shared secret key is generated based off the master encryption key and the channel name. The generated shared secret key will be used to encrypt the data before being offloaded to Pusher Channels. The shared secret is also sent as part of a successful authentication response as the client SDK will need to store it as it will be used for decrypting encrypted messages it receives. Again notice that since the encryption key never leaves your server, there is no way Pusher or any other person can read the messages if they don’t go through the authentication process - which is going to be done by the client side SDK.

NOTE: This shared secret is channel specific. For each channel subscribed to, a new shared secret is generated.

Here is a sample response:

1{
2      "auth": "3b65aa197f334949f0ef:ffd3094d43e1bb21d5eb849c3debcbba0f7dd32bddeb0bb7dd8441516029853d",
3      "channel_data": {
4        "user_id": "10",
5        "user_info": {
6          "random": "random"
7        }
8      },
9      "shared_secret": "oB4frIyBUiYVzbUSBFCBl7U5BxzW8ni6wIrO4UaYIeo="
10    }

Apart from privacy and security, another benefit encrypted channels provide is message authenticity and protection against forgery. So there is maximum guarantee that whatever message is being received was published by someone who has access to the encryption key.

Implementing encrypted channels

To show encrypted channels in practice, we will build a live feed application. The application will consist of a server and client. The server will be written in Go.

Before getting started, it will be nice to be aware of some limitations imposed by an encrypted channel. They are:

Channel name(s) must begin with private-encrypted-. Examples include private-encrypted-dashboard or private-encrypted-grocery-list. If you provide an encryption key but fail to follow the naming scheme, your data will not be encrypted.
Client events cannot be triggered
Channel and event names are not encrypted. This is for good reasons as events need to be dispatched to right clients and making sure an event in the Pusher Channels namespace - pusher: - cannot be used.

Before proceeding, you will need to create a new directory called pusher-encrypted-feeds. Make sure to create it within your $GOPATH. It can be done by issuing the following command in a terminal:

    $ mkdir pusher-encrypted-feeds

Prerequisites

Golang >=1.8
A Pusher account
OpenSSL tool.

If you are a Windows user, please note that you can make use of Git Bash since it comes with the OpenSSL toolkit.

Building the server

The first thing to do is to create a Pusher Channels account if you don’t have one already. You will need to take note of your app keys and secret as we will be using them later on in the tutorial.

In the pusher-encrypted-feeds directory, you will need to create another directory called server.

The next step of action is to create a .env file to contain the secret and key gotten from the dashboard. You should paste in the following contents:

1// pusher-encrypted-feeds/server/.env
2    
3    PUSHER_APP_ID="PUSHER_APP_ID"
4    PUSHER_APP_KEY="PUSHER_APP_KEY"
5    PUSHER_APP_SECRET="PUSHER_APP_SECRET"
6    PUSHER_APP_CLUSTER="PUSHER_APP_CLUSTER"
7    PUSHER_APP_SECURE="1"
8    PUSHER_CHANNELS_ENCRYPTION_KEY="PUSHER_CHANNELS_ENCRYPTION_KEY"

PUSHER_CHANNELS_ENCRYPTION_KEY will be the master encryption key used to generate the shared secret and it should be difficult to guess. It is also required to be a 32 byte encryption key. You can generate a suitable encryption key with the following command:

    $ openssl rand -base64 24

You will also need to install some dependencies - the Pusher Go SDK and another for parsing the .env file you previously created. You can grab those dependencies by running:

1$ go get github.com/joho/godotenv
2    $ go get github.com/pusher/pusher-http-go

You will need to create a main.go file and paste in the following content:

1// pusher-encrypted-feeds/server/main.go
2    
3    package main
4    
5    import (
6            "encoding/json"
7            "errors"
8            "flag"
9            "fmt"
10            "io/ioutil"
11            "log"
12            "net/http"
13            "os"
14            "strings"
15            "sync"
16            "time"
17    
18            "github.com/joho/godotenv"
19            pusher "github.com/pusher/pusher-http-go"
20    )
21    
22    func main() {
23    
24            port := flag.Int("http.port", 1400, "Port to run HTTP service on")
25    
26            flag.Parse()
27    
28            err := godotenv.Load()
29            if err != nil {
30                    log.Fatal("Error loading .env file")
31            }
32    
33            appID := os.Getenv("PUSHER_APP_ID")
34            appKey := os.Getenv("PUSHER_APP_KEY")
35            appSecret := os.Getenv("PUSHER_APP_SECRET")
36            appCluster := os.Getenv("PUSHER_APP_CLUSTER")
37            appIsSecure := os.Getenv("PUSHER_APP_SECURE")
38    
39            var isSecure bool
40            if appIsSecure == "1" {
41                    isSecure = true
42            }
43    
44            client := &pusher.Client{
45                    AppId:               appID,
46                    Key:                 appKey,
47                    Secret:              appSecret,
48                    Cluster:             appCluster,
49                    Secure:              isSecure,
50                    EncryptionMasterKey: os.Getenv("PUSHER_CHANNELS_ENCRYPTION_KEY"),
51            }
52    
53            mux := http.NewServeMux()
54    
55            f := &feed{
56                    mu:   &sync.RWMutex{},
57                    data: make(map[string]string, 0),
58            }
59    
60            mux.Handle("/feed", createFeedTitle(client, f))
61            mux.Handle("/pusher/auth", authenticateUsers(client))
62            
63            log.Println("Starting HTTP server")
64            log.Fatal(http.ListenAndServe(fmt.Sprintf(":%d", *port), mux))
65    }
66    
67    type feed struct {
68            data map[string]string
69    
70            mu *sync.RWMutex
71    }
72    
73    func (f *feed) exists(title string) bool {
74            f.mu.RLock()
75            defer f.mu.RUnlock()
76            _, ok := f.data[title]
77            return ok
78    }
79    
80    func (f *feed) Add(title, content string) error {
81            if f.exists(title) {
82                    return errors.New("title already exists")
83            }
84    
85            f.mu.Lock()
86            defer f.mu.Unlock()
87            f.data[title] = content
88            return nil
89    }
90    
91    const (
92            successMsg = "success"
93            errorMsg   = "error"
94    )
95    
96    func createFeedTitle(client *pusher.Client, f *feed) http.HandlerFunc {
97            return func(w http.ResponseWriter, r *http.Request) {
98                    w.Header().Set("Access-Control-Allow-Origin", "*")
99                    w.Header().Set("Access-Control-Allow-Methods", "POST, GET, OPTIONS, PUT, DELETE")
100                    w.Header().Set("Access-Control-Allow-Headers", "Accept, Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization")
101    
102                    if r.Method == http.MethodOptions {
103                            return
104                    }
105    
106                    writer := json.NewEncoder(w)
107    
108                    type respose struct {
109                            Message   string `json:"message"`
110                            Status    string `json:"status"`
111                            Timestamp int64  `json:"timestamp"`
112                    }
113    
114                    if r.Method != http.MethodPost {
115                            w.WriteHeader(http.StatusMethodNotAllowed)
116                            writer.Encode(&respose{
117                                    Message:   http.StatusText(http.StatusMethodNotAllowed),
118                                    Status:    errorMsg,
119                                    Timestamp: time.Now().Unix(),
120                            })
121    
122                            return
123                    }
124    
125                    var request struct {
126                            Title   string `json:"title"`
127                            Content string `json:"content"`
128                    }
129    
130                    if err := json.NewDecoder(r.Body).Decode(&request); err != nil {
131                            w.WriteHeader(http.StatusBadRequest)
132                            writer.Encode(&respose{
133                                    Message:   "Invalid request body",
134                                    Status:    errorMsg,
135                                    Timestamp: time.Now().Unix(),
136                            })
137                            return
138                    }
139    
140                    if len(strings.TrimSpace(request.Title)) == 0 {
141                            w.WriteHeader(http.StatusBadRequest)
142                            writer.Encode(&respose{
143                                    Message:   "Title field is empty",
144                                    Status:    errorMsg,
145                                    Timestamp: time.Now().Unix(),
146                            })
147                            return
148                    }
149    
150                    if len(strings.TrimSpace(request.Content)) == 0 {
151                            w.WriteHeader(http.StatusBadRequest)
152                            writer.Encode(&respose{
153                                    Message:   "Content field is empty",
154                                    Status:    errorMsg,
155                                    Timestamp: time.Now().Unix(),
156                            })
157                            return
158                    }
159    
160                    if err := f.Add(request.Title, request.Content); err != nil {
161                            w.WriteHeader(http.StatusAlreadyReported)
162                            writer.Encode(&respose{
163                                    Message:   err.Error(),
164                                    Status:    errorMsg,
165                                    Timestamp: time.Now().Unix(),
166                            })
167                            return
168                    }
169    
170                    go func() {
171    
172                            _, err := client.Trigger("private-encrypted-feeds", "items", map[string]string{
173                                    "title":     request.Title,
174                                    "content":   request.Content,
175                                    "createdAt": time.Now().String(),
176                            })
177    
178                            if err != nil {
179                                    fmt.Println(err)
180                            }
181    
182                    }()
183    
184                    w.WriteHeader(http.StatusOK)
185                    writer.Encode(&respose{
186                            Message:   "Feed item was successfully added",
187                            Status:    errorMsg,
188                            Timestamp: time.Now().Unix(),
189                    })
190            }
191    }
192    
193    func authenticateUsers(client *pusher.Client) http.HandlerFunc {
194            return func(w http.ResponseWriter, r *http.Request) {
195                    // Handle CORS
196                    w.Header().Set("Access-Control-Allow-Origin", "*")
197                    w.Header().Set("Access-Control-Allow-Methods", "POST, GET, OPTIONS, PUT, DELETE")
198                    w.Header().Set("Access-Control-Allow-Headers", "Accept, Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization")
199    
200                    if r.Method == http.MethodOptions {
201                            return
202                    }
203    
204                    params, err := ioutil.ReadAll(r.Body)
205                    if err != nil {
206                            w.WriteHeader(http.StatusBadRequest)
207                            return
208                    }
209    
210                    presenceData := pusher.MemberData{
211                            UserId: "10",
212                            UserInfo: map[string]string{
213                                    "random": "random",
214                            },
215                    }
216    
217                    response, err := client.AuthenticatePresenceChannel(params, presenceData)
218                    if err != nil {
219                            w.WriteHeader(http.StatusBadRequest)
220                            return
221                    }
222    
223                    w.Write(response)
224            }
225    }

In the above, we create an HTTP server with two endpoints:

/pusher/auth for authentication of client SDKs.
/feed for the addition of a new feed item.

Note that the feed items will not be stored in a persistent database but in memory instead

You should be able to run the server now. That can be done with:

    $ go run main.go

Building the client

The client is going to contain three pages:

a dashboard page
a form page for adding new feed items
a feed page for displaying feed items in realtime as received from the encrypted channel.

You will need to create a directory called client. That can be done with:

    $ mkdir client

To get started, we will need to build the form page to allow new items to be added. You will need to create a file called new.html with:

    $ touch new.html

In the newly created new.html file, paste the following content:

1<!-- pusher-encrypted-feeds/client/new.html -->
2    
3    <!DOCTYPE html>
4    <html>
5      <head>
6        <meta charset="utf-8">
7        <meta name="viewport" content="width=device-width, initial-scale=1">
8        <title>Pusher realtime feed</title>
9        <meta name="viewport" content="width=device-width, initial-scale=1" />
10        <link rel="icon" type="image/x-icon" href="favicon.ico" />
11        <base href="/" />
12        <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/bulma/0.7.2/css/bulma.min.css">
13    <style>
14    .hidden { display: none }
15    </style>
16      <body>
17          <section class="section">
18              <div class="container">
19    <div class="columns">
20      <div class="column is-5">
21        <h3 class="notification">Create a new post</h3>
22        <div class="notification is-success hidden" id="success"></div>
23        <div class="is-danger notification hidden" id="error"></div>
24        <form id="feed-form">
25          <div class="field">
26            <label class="label">Title : </label>
27            <div class="control">
28              <input
29                class="input"
30                type="text"
31                placeholder="Post title"
32                name="title"
33                id="title"
34              />
35            </div>
36          </div>
37    
38          <div><label>Message: </label></div>
39          <div>
40            <textarea
41              rows="10"
42              cols="70"
43              name="content"
44                 id="content"
45            ></textarea>
46          </div>
47    
48    
49    <button id="submit" class="button is-info">
50      Send
51    </button>
52        </form>
53              </div>
54      <div class="is-7"></div>
55           </section>
56      </body>
57      <script src="app.js"></script>
58    </html>

This is as simple as can be. We reference the Bulma css library, we create a form with an input and text field. Finally we link to a non-existent file called app.js - we will create that in a bit.

To view what this file looks like, you should navigate to the client directory and run the following command:

    $ python -m http.server 8000

Here I used Python’s inbuilt server but you are free to use whatever.

You should visit localhost:8000/new.html . You should be presented with something similar to the image below:

As said earlier, we linked to a non-existent file app.js, we will need to create it and fill it with some code. Create the app.js file with:

    $ touch app.js

In the newly created file, paste the following:

1// pusher-encrypted-channels/client/app.js
2    
3    (function() {
4      const submitFeedBtn = document.getElementById('feed-form');
5      const isDangerDiv = document.getElementById('error');
6      const isSuccessDiv = document.getElementById('success');
7    
8      if (submitFeedBtn !== null) {
9        submitFeedBtn.addEventListener('submit', function(e) {
10          isDangerDiv.classList.add('hidden');
11          isSuccessDiv.classList.add('hidden');
12          e.preventDefault();
13          const title = document.getElementById('title');
14          const content = document.getElementById('content');
15    
16          if (title.value.length === 0) {
17            isDangerDiv.classList.remove('hidden');
18            isDangerDiv.innerHTML = 'Title field is required';
19            return;
20          }
21    
22          if (content.value.length === 0) {
23            isDangerDiv.classList.remove('hidden');
24            isDangerDiv.innerHTML = 'Content field is required';
25            return;
26          }
27    
28          fetch('http://localhost:1400/feed', {
29            method: 'POST',
30            body: JSON.stringify({ title: title.value, content: content.value }),
31            headers: {
32              'Content-Type': 'application/json',
33            },
34          }).then(
35            function(response) {
36              if (response.status === 200) {
37                isSuccessDiv.innerHTML = 'Feed item was successfully added';
38                isSuccessDiv.classList.remove('hidden');
39                setTimeout(function() {
40                  isSuccessDiv.classList.add('hidden');
41                }, 1000);
42                return;
43              }
44    
45              if (response.status === 208) {
46                message = 'Feed item already exists';
47              } else {
48                message = response.statusText;
49              }
50    
51              isDangerDiv.innerHTML = message;
52              isDangerDiv.classList.remove('hidden');
53            },
54            function(error) {
55              isDangerDiv.innerHTML = 'Could not create feed item';
56              isDangerDiv.classList.remove('hidden');
57            }
58          );
59        });
60      }
61    })();

In the above, we validate the form whenever the Send button is clicked. If the form contains valid data, it is sent to the Go server for processing. The server will store it and trigger a message to Pusher Channels.

Go ahead and submit the form. If successful and you are on the Debug Console, you will notice something of the following sort:

The next point of action will be to create the feeds page so entries can be viewed in realtime. You will need to create a file called feed.html. That can be done with:

    $ touch feed.html

In the new file, paste the following HTML code:

1<!-- pusher-encrypted-channels/client/feed.html -->
2    
3    <!DOCTYPE html>
4    <html>
5      <head>
6        <meta charset="utf-8">
7        <meta name="viewport" content="width=device-width, initial-scale=1">
8        <title>Pusher realtime feed</title>
9        <meta name="viewport" content="width=device-width, initial-scale=1" />
10        <link rel="icon" type="image/x-icon" href="favicon.ico" />
11        <base href="/" />
12        <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/bulma/0.7.2/css/bulma.min.css">
13      <body>
14          <section class="section">
15              <div class="container">
16           <h1 class="notification is-info">Your feed</h1>
17    <div class="columns">
18      <div class="column is-7">
19        <div id="feed">
20        </div>
21      </div>
22    </div>
23              </div>
24           </section>
25      </body>
26      <script src="https://js.pusher.com/4.3/pusher.min.js"></script>
27      <script src="https://cdnjs.cloudflare.com/ajax/libs/handlebars.js/4.1.0/handlebars.min.js"></script>
28      <script src="app.js"></script>
29    </html>

This page is basically empty. It will be updated by the Channels client SDK as it receives data. We are linking to the Pusher Channels client SDK and Handlebars. Handlebars is used to compile templates we will inject into the page.

To be able to receive and update the feeds page with data the app.js file has to be updated to make use of Pusher Channels. In app.js , append the following code:

1// pusher-encrypted-feed/client/app.js
2    
3    // Sample template to be injected
4    const tmpl = `
5          <div class="box">
6            <article class="media">
7              <div class="media-left">
8                <figure class="image is-64x64">
9                  <img src="https://bulma.io/images/placeholders/128x128.png" alt="Image" />
10                </figure>
11              </div>
12              <div class="media-content">
13                <div class="content">
14                  <p>
15                    <strong>{{title}}</strong>
16                    <small>{{createdAt}}</small> <br />
17                    {{content}}
18                  </p>
19                </div>
20              </div>
21            </article>
22          </div>
23    `;
24    
25      const APP_KEY = 'PUSHER_APP_KEY';
26      const APP_CLUSTER = 'PUSHER_CLUSTER';
27    
28      Pusher.logToConsole = true;
29    
30      const pusher = new Pusher(APP_KEY, {
31        cluster: APP_CLUSTER,
32        authEndpoint: 'http://localhost:1400/pusher/auth',
33      });
34    
35      const channel = pusher.subscribe('private-encrypted-feeds');
36      // Use Handlebars to compile the template
37      const template = Handlebars.compile(tmpl);
38      const feedDiv = document.getElementById('feed');
39    
40      channel.bind('items', function(data) {
41        // replace some fields in the template with data from the event.
42        const html = template(data);
43    
44        const divElement = document.createElement('div');
45        divElement.innerHTML = html;
46    
47        // Update the page
48        feedDiv.appendChild(divElement);
49      });

Remember to replace both PUSHER_CLUSTER and PUSHER_KEY with your credentials

With the addition above, the entire app.js should look like:

1// pusher-encrypted-feed/client/app.js
2    
3    (function() {
4      const submitFeedBtn = document.getElementById('feed-form');
5      const isDangerDiv = document.getElementById('error');
6      const isSuccessDiv = document.getElementById('success');
7    
8      if (submitFeedBtn !== null) {
9        submitFeedBtn.addEventListener('submit', function(e) {
10          isDangerDiv.classList.add('hidden');
11          isSuccessDiv.classList.add('hidden');
12          e.preventDefault();
13          const title = document.getElementById('title');
14          const content = document.getElementById('content');
15    
16          if (title.value.length === 0) {
17            isDangerDiv.classList.remove('hidden');
18            isDangerDiv.innerHTML = 'Title field is required';
19            return;
20          }
21    
22          if (content.value.length === 0) {
23            isDangerDiv.classList.remove('hidden');
24            isDangerDiv.innerHTML = 'Content field is required';
25            return;
26          }
27    
28          fetch('http://localhost:1400/feed', {
29            method: 'POST',
30            body: JSON.stringify({ title: title.value, content: content.value }),
31            headers: {
32              'Content-Type': 'application/json',
33            },
34          }).then(
35            function(response) {
36              if (response.status === 200) {
37                isSuccessDiv.innerHTML = 'Feed item was successfully added';
38                isSuccessDiv.classList.remove('hidden');
39                setTimeout(function() {
40                  isSuccessDiv.classList.add('hidden');
41                }, 1000);
42                return;
43              }
44    
45              if (response.status === 208) {
46                message = 'Feed item already exists';
47              } else {
48                message = response.statusText;
49              }
50    
51              isDangerDiv.innerHTML = message;
52              isDangerDiv.classList.remove('hidden');
53            },
54            function(error) {
55              isDangerDiv.innerHTML = 'Could not create feed item';
56              isDangerDiv.classList.remove('hidden');
57            }
58          );
59        });
60      }
61    
62      const tmpl = `
63          <div class="box">
64            <article class="media">
65              <div class="media-left">
66                <figure class="image is-64x64">
67                  <img src="https://bulma.io/images/placeholders/128x128.png" alt="Image" />
68                </figure>
69              </div>
70              <div class="media-content">
71                <div class="content">
72                  <p>
73                    <strong>{{title}}</strong>
74                    <small>{{createdAt}}</small> <br />
75                    {{content}}
76                  </p>
77                </div>
78              </div>
79            </article>
80          </div>
81    `;
82    
83      const APP_KEY = 'PUSHER_APP_KEY';
84      const APP_CLUSTER = 'PUSHER_CLUSTER';
85    
86      Pusher.logToConsole = true;
87    
88      const pusher = new Pusher(APP_KEY, {
89        cluster: APP_CLUSTER,
90        authEndpoint: 'http://localhost:1400/pusher/auth',
91      });
92    
93      const channel = pusher.subscribe('private-encrypted-feeds');
94      const template = Handlebars.compile(tmpl);
95      const feedDiv = document.getElementById('feed');
96    
97      channel.bind('items', function(data) {
98        const html = template(data);
99    
100        const divElement = document.createElement('div');
101        divElement.innerHTML = html;
102    
103        feedDiv.appendChild(divElement);
104      });
105    })();

You can go ahead to open the feed.html page on a tab and new.html in another. Watch closely as whatever data you submit in new.html appears in feed.html. You can also keep an eye on the Debug Console to make sure all data is encrypted.

To make this app a little more polished, add an index.html page. You can find the source code at the accompanying GitHub repository of this tutorial.

Conclusion

In this tutorial, I introduced you to a lesser known feature of Pusher Channels - end to end encryption with encrypted channels. We also built an application that uses encrypted channels instead of the regular public channels you might be used to.

As always, the entire code for this article can be found on GitHub.

Docs

Tutorials

Support

Glossary

End to end encryption in Go with Pusher Channels

Introduction

Understanding encrypted channels

Implementing encrypted channels

Prerequisites

Building the server

Building the client

Conclusion

Products

Developers

Company

Legal