Continuous Dashboarding - Your DevOps Airbag

name: inverse
class: center, middle, inverse
layout: true
<div class="header"><span>@unixbigot — Christopher Biggs — Continuous Dashboarding</div>
<div class="footer"><span>Linux.conf.au Sysadmin Miniconf Jan 2017</span></div>
---
layout: true
<div class="header"><span>@unixbigot — Christopher Biggs — Continuous Dashboarding</div>
<div class="footer"><span>Linux.conf.au Sysadmin Miniconf Jan 2017</span></div>
---
class: center, middle
template: inverse

# Continuous Dashboarding - Your DevOps Airbag

.green[Building and shipping Agile dashboards using your existing data silos.]

![](node-red-title-flow.png)

.bottom.right[
Christopher Biggs
<br>.blue[@unixbigot]
]

---
.left-column[
# Agenda
]
.right-column[
### **Strategy**:
* .red[**Who**] are your dashboards for?
* .orange[**What**] to monitor
* .yellow[**When**] to build your dashboards

### **Tactics**:
* .green[**How**] to develop dashboards
* .blue[**Where**] to fit dashboards into your DevOps pipeline
* .purple[**With**] some suggested tools 
]

---
class: center, middle
template: inverse

# Who?

## Dashboards and Business Intelligence (BI) are .red[Product]

### Think about who your .green[customer] is

---
.left-column[
#Strategy
## Who
]
.right-column.autodim[
## Who are your dashboards for?

#### Dashboards are traditionally targeted at ops.
 * Measure **all** the things!
 * Wake me if it's on **fire**
 * Little feedback goes back **upstream**

.img-800w[![Munin](munin-boring.png)]
]

---
.left-column[
#Strategy
## Who
]
.right-column.autodim[
## Who are your dashboards for?

#### Traditionally targeted at ops, but
#### DevOps is a thing.
 * Development is a different **point of view**
 * **Instrument** your applications
 * Measure **trends** and **levels** in your KPIs
 * Look for **out-of-range** metrics
 * What does normal **look** like?
 * Regression-test your **bottom line**
]

---

.left-column[
#Strategy
## Who
]
.right-column.autodim[
## Who are your dashboards for?

#### Traditionally targeted at ops, but
#### DevOps is a thing
#### Support and Sales need to know everything's OK, too
 * Is the first time you know of problems when support escalates?
 * Do support call ops and ask "Is everything OK, we're getting a lot
of calls"?
 * "Well Mrs Arbuckle, I see other customers are having problems too"
]

---
.left-column[
#Strategy
## Who
]
.right-column.autodim[
## Who are your dashboards for?

#### Traditionally targeted at ops, but
#### DevOps is a thing
#### Support and Sales need to know everything's OK too
#### Management want reassurance (and to know you deliver)
 * The morning report for "How long till my next **Porsche**"
 * Shipped a great new feature?  You now have a **hotline** to the CEOs eyeballs.
]

---
.left-column[
#Strategy
## Who
]
.right-column.autodim[
## Who are your dashboards for?

#### Traditionally targeted at ops, but
#### DevOps is a thing
#### Support and Sales need to know everything's OK too
#### Management want reassurance (and to know you deliver)
#### Customers (do you dare?)
 * What are you **ashamed** of them seeing?
 * What could you **do** about that?

]
---
.left-column[
#Strategy
## Who
]
.right-column.autodim[
## Give **customers** a weather report

#### Take notice of load spikes
 * "We are processing your order.   This hour that
takes around 25 seconds but could be up to two minutes"
]

---
.left-column[
#Strategy
## Who
]
.right-column.autodim[
## Give **customers** a weather report

#### Take notice of load spikes
#### Respond to temporary outages
 * "Sorry, it looks like PayPal has been experiencing high error rates
 for the last 15 minutes.  May we suggest you try AliPay or Credit
Card?"
   * This could head off a flood of support calls or bad reviews
]
---
.left-column[
#Strategy
## Who
]
.right-column.autodim[
## Give **customers** a weather report

#### Take notice of load spikes
#### Respond to temporary outages
#### Assure customers that errors are being acted on
 * "We're sorry, this page experienced a problem.  Bug #3342 has
already been forwarded to our Dev Team.   You'll get an email when
it's fixed, or check our .fakelink[change log]"
   * again, turn your failures into goodwill
]
---
.left-column[
#Strategy
## Who
]
.right-column.autodim[
## What's relevant to this **support** call?

#### Notice recent errors affecting this customer
 * "Mr Lee encountered .fakelink[Bug #3342] six minutes ago"

]
---
.left-column[
#Strategy
## Who
]
.right-column.autodim[
## What's relevant to this **support** call?

#### Notice recent errors affecting this customer
#### Notice recent outages affecting this customer
 * "Currently experiencing elevated transaction declines for Westpac
cards"

]
---
.left-column[
#Strategy
## Who
]
.right-column.autodim[
## What's relevant to this **support** call?

#### Notice recent errors affecting this customer
#### Notice recent outages affecting this customer
#### Remember recent behaviour of this customer
 * "Dr Patel had her password reset yesterday, and presented her old
 password three times today"

]
---
class: center, middle
template: inverse

# What?

## Not just **things**, also .green[patterns] and .red[trends]

---
.left-column[
#Strategy
## Who
## What
]
.right-column.autodim[
## What should I be monitoring?

#### Traditionally - **Things**
  * system load indicators
  * rates of key events (pageloads, signups, checkouts etc.)
  * service status
  * error alerts ("pages")
]
---
.left-column[
#Strategy
## Who
## What
]
.right-column.autodim[
## What should I be monitoring?

#### Traditionally - **Things**
#### Now - **Patterns** and **Trends**
  * Business Goals - Rates plus Trends
]
---
.left-column[
#Strategy
## Who
## What
]
.right-column.autodim[
## What should I be monitoring?

#### Traditionally - **Things**
#### Now - **Patterns** and **Trends**
  * Business Goals - Rates plus Trends
  * Unavoidable errors - look for out-of-range levels
]
---
.left-column[
#Strategy
## Who
## What
]
.right-column.autodim[
## What should I be monitoring?

#### Traditionally - **Things**
#### Now - **Patterns** and **Trends**
  * Business Goals - Rates plus Trends
  * Unavoidable errors - look for out-of-range levels
  * Location and traffic patterns
  * User Experience metrics
]
---
.left-column[
#Strategy
## Who
## What
]
.right-column.autodim[
## What should I be monitoring?

#### Traditionally - **Things**
#### Now - **Patterns** and **Trends**
  * Business Goals - Rates plus Trends
  * Unavoidable errors - look for out-of-range levels
  * Location and traffic patterns
  * User Experience metrics
  * Health of your third-party interfaces
]
---
.left-column[
#Strategy
## Who
## What
]
.right-column.autodim[
## What should I be monitoring?

#### Traditionally - **Things**
#### Now - **Patterns** and **Trends**
  * Business Goals - Rates plus Trends
  * Unavoidable errors - look for out-of-range levels
  * Location and traffic patterns
  * User Experience metrics
  * Health of your third-party interfaces
  * App and product reviews (iTunes, Google, Yelp, Amazon...)
]
---
.left-column[
#Strategy
## Who
## What
]
.right-column.autodim[
## What should I be monitoring?

#### Traditionally - **Things**
#### Now - **Patterns** and **Trends**
  * Business Goals - Rates plus Trends
  * Unavoidable errors - look for out-of-range levels
  * Location and traffic patterns
  * User Experience metrics
  * Health of your third-party interfaces
  * App and product reviews (iTunes, Google, Yelp, Amazon...)
  * Social media (facebook, twitter, et.al.)
]
---
class: center, middle
template: inverse

# When?

## Make Business Intelligence part of your .orange[**DevOps**] processes

---
.left-column[
#Strategy
## Who
## What
## When
]
.right-column.autodim.tight[
## When should I build dashboards?
#### Early and Often!
* Dashboards inform your planning and strategy
* Make sure you *really* know the status quo before you code
]
---
.left-column[
#Strategy
## Who
## What
## When
]
.right-column.autodim.tight[
## When should I build dashboards?

#### Early and Often!

#### Continuously: Dashboards are (subjective) tests
* **Hopefully** you already run tests before you ship
* Think about what tests you should run **continuously**
]
---
.left-column[
#Strategy
## Who
## What
## When
]
.right-column.autodim.tight[
## When should I build dashboards?

#### Early and Often!
#### Continously: Dashboards are (subjective) tests
#### Releases: Dashboards are longitudinal tests
* How does this release compare to last release?
]
---
.left-column[
#Strategy
## Who
## What
## When
]
.right-column.autodim.tight[
## When should I build dashboards?

#### Early and Often!
#### Continuously: Dashboards are (subjective) tests
#### Releases: Dashboards are longitudinal tests
#### Fridays: Dashboards are your DevOps "airbag"
* Is the bucket still under the money waterfall?
* Is it Friday?   Does that scare you?
]
---
.left-column[
#Strategy
## Who
## What
## When
]
.right-column.autodim.tight[
## When should I build dashboards?

#### Early and Often!
#### Continuously: Dashboards are (subjective) tests
#### Releases: Dashboards are longitudinal tests
#### Fridays: Dashboards are your DevOps "airbag"
#### Coding: Dashboards help you think about Features holistically
* Who needs to know about this new feature?   Training?
* What other parts of our ecosystem will this affect?
]
---
.left-column[
#Strategy
## Who
## What
## When
]
.right-column.autodim.tight[
## When should I build dashboards?

#### Early and Often!
#### Continuously: Dashboards are (subjective) tests
#### Releases: Dashboards are longitudinal tests
#### Fridays: Dashboards are your DevOps "airbag"
#### Coding: Dashboards help you think about Features holistically
#### During test: Dashboards help you plan for adverse outcomes
* **Validate** your assumptions
* What do I **do** if this breaks?
* Who will notice a failure?   Who needs to be **notified**?
]
---
class: center, middle
template: inverse

# How?

## Encourage a .purple[**data-loving**] culture

---
.left-column[
#Tactics
## Who
## What
## When
## How
]
.right-column.bulletsh4[
## So, How do I get there from here?
#### **Experiment** with your data
#### **Develop** dashboards alongside features
#### **Test** your dashboards
#### **Deploy** your BI configuration as build artifacts.
]
---
.left-column[
#Tactics
## Who
## What
## When
## How
]
.right-column.autodim.tight[
## So, How do I get there from here?
#### **Experiment** with your data
* Derive synergy from bringing your silos together
* Visual dataflow tools like .red[**Node-RED**] ([nodered.org](http://nodered.org/))
* Rapid dashboard tools like .green[**Blynk**] ([blynk.cc](http://blynk.cc))
* Explore your data with GUI tools like .purple[**Kibana**] ([elastic.co](http://elastic.co/))

.img-320w[![Node-RED](nodered-frontdoor.png)]
.img-200h[![Blynk](blynk_bi_goals.png)]
.img-320w[![Kibana](kibana-basics.jpg)]

]
---
.left-column[
#Tactics
## Who
## What
## When
## How
]
.right-column.autodim[
## So, How do I get there from here?
#### **Experiment** with your data
#### **Develop** dashboards alongside features
* Orchestrate your BI stack (Docker, Vagrant, whatever)
* Choose tools that produce shippable artifacts
* Demonstrate your dashboards as a deliverable
]
---
.left-column[
#Tactics
## Who
## What
## When
## How
]
.right-column.autodim[
## So, How do I get there from here?
#### **Experiment** with your data
#### **Develop** dashboards alongside features
#### **Test** your dashboards against your code
* ...and vice-versa
* Dashboards are **Customer-Facing** product.
* Trustworthiness is crucial
]
---
class: center, middle
template: inverse

# Where?

## Dashboards as .blue[**code**]

---
.left-column[
#Tactics
## Who
## What
## When
## How
## Where
]
.right-column.autodim[
## Inserting dashboards into your development pipeline
#### Code-review your Dashboards
* Pretty-print JSON or XML data files
* Use Visual diffing tools to highlight changes (eg. pdiffy)
]
---
.left-column[
#Tactics
## Who
## What
## When
## How
## Where
]
.right-column.autodim[
## Inserting dashboards into your development pipeline
#### Code-review your Dashboards
#### Part of your CI test runs
* Yes, you can unit-test dashboards
* Leverage your integration tests to produce data
* Behavioural testing tools to confirm UI outcomes
]
---
.left-column[
#Tactics
## Who
## What
## When
## How
## Where
]
.right-column.autodim[
## Inserting dashboards into your development pipeline
#### Code-review your Dashboards
#### Part of your CI test runs
#### Performance testing - daily/sprintly test
* Build a test that stresses your product to breaking point
* Confirm that the right alarms go off
* Break things, then verify your ecosystem responds
]
---
.left-column[
#Tactics
## Who
## What
## When
## How
## Where
]
.right-column.autodim[
## Inserting dashboards into your development pipeline
#### Code-review your Dashboards
#### Part of your CI test runs
#### Performance testing - daily/sprintly test
#### Automate deployment - easier with some tools than others
* Best case - install configuration files as ordinary artifacts 
* APIs - pull and push
* Database - dump/restore
* Web scraping - here be dragons
]
---

class: center, middle
template: inverse

# With

## Case Study: Adding value to .yellow[**Logstash**] with .red[**Node-RED**] and .green[**Blynk**]

---

.left-column[
#Tactics
## Who
## What
## When
## How
## Where
## With
]
.right-column.tight[
## .green[Elasticsearch], .yellow[Logstash], and .purple[Kibana] (ELK)

* .green[**Elasticsearch**] Fast Distributed NoSQL database
* .yellow[**Logstash**] (et.al) Push event streams, application logs, performance data and network stats to an Elasticsearch storage cluster
* .purple[**Kibana**] Query, Visualise and Dashboard your data
* ELK is Open Source: self-host, AWS, or SaaS
* Off-the-shelf docker image suitable for dev, CI and production
* Quick interactive querying
* Interactively build visulisations and dashboards
* Good support for import/export of dashboard components
* Weak on certain kinds of analysis and alerting
]
---
class: center, middle

.img-1000w[![Kibana](kibana-basics.jpg)]

---

.left-column[
#Tactics
## Who
## What
## When
## How
## Where
## With
]
.right-column.tight[
## .red[Node-RED] - A Visual system for Wiring the Internet

* Open source (Node.js)
* Originated at IBM, now under JS Foundation
* .red[**Flow**] based data processing
 - "Fire all the programmers", at last?
* Create flows using drag+drop flow editor
 - `http://nodered.example.com:1800/`
* Huge plugin ("node") ecosystem 
* Integrated dashboarding
 - `http://nodered.example.com:1800/ui`
 - or output values to other data stores

]
---

## Business Goal Monitor

.img-420h[![Business Goal Monitor](bi_goals.png)]
.img-420h[![Business Goal Dashboard](blynk_bi_goals.png)]

---

.left-column[
#Tactics
## Who
## What
## When
## How
## Where
## With
]
.right-column.small.tight[
## .red[Node-RED] - Nodes and Flows
* The .blue[**coloured boxes**] are .red[**Nodes**]
 * Nodes are **npm** modules
* Drag and drop nodes.  Connect with links to form .green[**Flows**]
* .green[**Flows**] persist as JSON data (`~/.node-red/flows.js`)
* .green[**Flows**] process .purple[**Messages**] (JSON)
* Over 750 node modules available
 * plus a general-purpose **function** node
 * or write-yer-own nodes

.img-640w[![Simplest flow example](simplest_flow.png)]
]
---
.left-column[
#Tactics
## Who
## What
## When
## How
## Where
## With
]
.right-column.tight.small[
## (ab)Using .red[Node-RED] for Business Dashboarding
* Search most SQL and NoSQL databases (eg. .yellow[**Elasticsearch**])
* Filter, Smooth, Report-by-Exception
* Built in HTML .purple[**dashboard**] nodes (`node-red-dashboard`)
* Push data to .green[**Blynk**] for iPhone/Android Apps

.img-320h[![Dashboard UI](dashboarde.png)]
.img-320h[![Server Error Dashboard](blynk_bi_errors.png)]

]
---
.left-column[
#Tactics
## Who
## What
## When
## How
## Where
## With
]
.right-column.small.tight[
## Blynk - Agile dashboarding for one user or a thousand
* .blue[blynk.cc] - cloud data, or self-host
* interactive dashboard editor
* share an app (with data) or clone (separate data)
* free for ad-hoc sharing, pay for app-store distribution

.img-320h[![Blynk Widgets](blynk_widgets.png)]
.img-320h[![Blynk Sharing](blynk_share.png)]
.img-320h[![Blynk Cloning](blynk_clone.png)]
]
---

## Server Error Monitor

.img-420h[![Server Error Monitor](bi_errors.png)]
.img-420h[![Server Error Dashboard](blynk_bi_errors.png)]

---
.left-column[
#Tactics
## Who
## What
## When
## How
## Where
## With
]
.right-column[
## Social Media Integration

* Receive feeds from twitter, facebook etc.
* Recieve feeds from app stores
* Post to slack, and monitor channels
* Analyse, Summarise, Respond, Alert
]
---
.left-column[
# Agenda
## Who
## What
## When
## How
## Where
## With
## .red[Summary]
]
.right-column.bulletsh4[
## Summary
#### Target your dashboards, keep them focused
#### You can do a lot more than just display graphs
#### Design your dashboards early
#### Be Agile: experiment, build, ship, iterate.
#### Dashboards are code
#### Bring all your data together for synergy
]
---
.left-column[
# Agenda
## Who
## What
## When
## How
## Where
## With
## .red[Summary]
]
.right-column.small.bulletsh4[
## Resources, Questions

#### My BrisJS talks on Node-RED + Blynk in more depth:
  - Videos at
  [YouTube](https://www.youtube.com/watch?v=MoP8zH2hbnY&t=151s) (more
  to come)
  - Slides at http://christopher.biggs.id.au/

#### Node-RED - [nodered.org](https://node-red.org/)
#### Blynk - [blynk.cc](https://blynk.cc/)

#### Me - Christopher Biggs
  - Twitter: .blue[@unixbigot]
  - Email: .blue[christopher@biggs.id.au]
  - Slides, and getting my advice: http://christopher.biggs.id.au/
]