Posts

Showing posts from 2016

Ubuntu AWS Change Screen Resolution (size)

sudo gedit /etc/vncserver/vncservers.conf VNCSERVERS="1:dev" #VNCSERVERARGS[1]="-geometry 3840x2160 -depth 24" VNCSERVERARGS[1]="-geometry 1680x1050 -depth 24"

PII, Safe Harbor, Privacy Shield

Safe Harbor is dead, long live the privacy shield

Kubernetes Health Checks

Kubernetes Health Checks

Password Reset Guidelines

Password Reset Guidelines

Analytics Cloud Db

Analytics Cloud Db

Git Branching Strategy

Git Branching Strategy

Graceful shutdown

Deep discussion on graceful shutdown Kubernetes Node.js

10+ million connections to server

High Scalability The Secret To 10 Million Concurrent Connections -The Kernel Is The Problem, Not The Solution Scaling to 12 Million Concurrent Connections: How MigratoryData Did It

OWL, RDF, SPARQL

Web Ontology Language OWL Resource Description Framework SPARQL query language for RDF

JustOne Data Lake

JustOne Data Lake - new storage for postgres

Speed up Ubuntu in Vbox

Speed up Ubuntu in Vbox

Interesting Analysis of SCRUM

The Author discusses some aspects of SCRUM that he doesn't prefer in this blog post .

Javascript json compare lib

The deep-diff lib compares two json structures and indicates the differences.

Database optimizer statistics (dates)

Optimizing statistics for dates

Elasticsearch query builder

Mirage

Netflix Caching

EV Cache

Sequence Diagram Tool

Web declarative sequence diagram tool

Microservices Website

Composable webpages

Backlog Vision to Value

Backlog Vision to Value 7 Product dimensions

IDs UUID

64 bit sortable sharded IDs - Instagram style

Charts

Visualizations D3js charts Google charts Chartjs Chartist n3-charts Ember charts Smoothie charts Chartkick ZingCharts Highcharts js Fusioncharts Flot amCharts EJS Chart uvCharts Plotly js VictoryPie Charts  (react)

Apache Drill vs Facebook Presto

Drill vs Presto SQL query across disparate data, sql, noSql, files, S3, etc.

Docker base image

Alpine really small base image

Kubernestes persistent volumes

kubernetes persisten volumes glusterfs filesystem which can provide persistent volumes for kubernetes docker images

vector clocks

vector clocks

Solr index and query nested documents

Solr nested documents Elasticsearch has better support for nested documents than Solr.

Microservices Distributed Transactions

Microservices Distributed Tansactions

Web Sequence Diagrams - Declarative

Web Sequence Diagrams

Solr generic schema tuple based

Elasticsearch generic schema tuple based How do they do this Solr nesting indexing and querying

Cassandra aggregations/sums

Cassandra aggregations/sums

Microservices best practices

Microservices best practices

Java Serializer Performance

Java Serializer Performance

SSL resume

SSL resume

Indeed recommendation engine

Indeed Recommendation Engine

Very Awesome Microservices Platform (VAMP)

VAMP

Data Analytics Platforms

http://www.zoomdata.com/product/embedded-data-analytics-visualization/ https://www.bottlenose.com/products http://www.striim.com/products/ Lucidworks Fusion Datastax DSE

upstart script (tiny)

Simple upstart script env PORT=5001 start on runlevel [2345] stop on runlevel [016] respawn setuid john chdir /home/john/Applications/my-nodejs-server/ exec node index.js Can configure environment variables in a faux  12 factor  style. NodeJS app will automatically start on boot, and stop on shutdown. Upstart will restart the app if it crashes. Runs as unpriveleged user 'john'. Upstart (1.4 and above) logs all output to  /var/log/upstart/my-nodejs-server.log  which will be rotated by logrotate. Can manage the lifecycle with  sudo service my-nodejs-server {start,stop,restart} . Can temporarily disable the app from starting on boot with an override file:  $ echo manual | sudo tee /etc/init/my-nodejs-server.override .

SocketListener

import java.io.IOException; import java.io.InputStream; import java.net.Inet4Address; import java.net.ServerSocket; import java.net.Socket; import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; /**  * Socket listener for unit test to listen for messages.  *  * @author dev  *  */ public class SocketListener { String host; int port; ServerSocket s; ExecutorService executor = Executors.newSingleThreadExecutor(); public SocketListener(String host, int port) { this.host = host; this.port = port; } public SocketListener init() { try { s = new ServerSocket(port, 10, Inet4Address.getByName(host)); Worker worker = new Worker(); executor.submit(worker); } catch (IOException e) { throw new RuntimeException("Failed to listen", e); } return this; } public void close() { if (s != null) { try { s.close(); } catch (IOException e) { // ignore } } } public class

Browsing ADFS

connecting to the AD domain controller via ldap using Apache Directory Studio. No special access required. Same as how your user authenticates on OS. ldap://10.10.10.10:636 (bind as your domain account) Search for user: Base DN: DC=youDomain,DC=smtf,DC=ds Windows account id: (sAMAccountName=tcollinsworth) Name: (displayName=Troy*)

WebHook testing

RequestBin

Stacktrace of Java process

jstack To quickly dump stack from command line for eclipse process jps | grep RemoteTestRunner | cut -f 1 -d ' ' | xargs jstack

Stacktrace of Java process

jstack To quickly dump stack from command line for eclipse process jps | grep RemoteTestRunner | cut -f 1 -d ' ' | xargs jstack

Force IPv4

-Djava.net.preferIPv4Stack=true -Djava.net.preferIPv6Addresses=false

Java 8 UTC datetime as string

public static String getUTCDatetimeAsString() {    return LocalDateTime.now(Clock.systemUTC()); }

Code monitoring and continuous inspection

SonarQube

General RPC

gRPC

Top Web Sites Technology

Technology used in most popular websites Most used: Javascript 100% MySQL / MariaDB 53% Java 47% C++ 33% Python 27%

Pagination and infinite scrolling

Pagination and infinite scrolling

System orchestration - Terraform

Terraform

System monitoring and metrics

Prometheus.io

JWT

JWT benefits and drawbacks Where to store local storage or cookies? Local storage is easy to compromise. Use cookies and ensure XSS and CSRF protections are implemented.

Web Analytics - open source

PIWIK

X remote display

To eliminate logging errors: unset XDG_RUNTIME_DIR To run apps and display remotely $ DISPLAY=:1 gedit or export DISPLAY=:1 gedit

Ubuntu open new tab in same terminal

Need xdotool WID= xprop -root | grep "_NET_ACTIVE_WINDOW(WINDOW)"| awk '{print $5}'; xdotool windowfocus $WID; xdotool key ctrl+shift+t $WID

Android Studio on Ubuntu Look & Feel

StackOverflow download a color scheme for IntelliJ such as "Classic Eclipse" from http://www.ideacolorthemes.org/home/ It a jar file then you do  File > Import Settings...  and open the jar file and restart. You should also go into  Preferences > Keymap  and change to the dropdown to be eclipse sortcuts so that you can get about easier.

Internet tunnel to localhost

ngrok expose a local server behind a NAT or firewall to the internet

BFF Architecture

Backends For Frontends written by Sam Newman author of Building of Microservices

Auth0 and Tyk.io API Gateway Integration

In Tyk: Create an API - mocks run before authentication, so use virtual end-point if you are mocking the API for testing Create Policy, select API under Access Rights, save and record Policy ID for entering in Auth0 Create a unique tag on the policy so that it can be queried to validate it is being used by Tyk (chicken & egg here as you have to create API before policy for rights and need to enter policy ID in API Auth) Set API Authentication Mode to JWT, HMAC, and set the secret to some plain text Set API Authentication Identity Source to sub or it will be logged that base wasn't found, using sub Set API, Policy Field Name to pol or some other unused JWT claim name This will be used to find the Tyk Policy by ID (not name) which is auto generated when the policy is created In Auth0: Set the secret to some base64 encoded plain text since Auth0 requires base64 & Tyk requires plain text In Rules : Settings add a key=value, e.g., auth0 clientId=Tyk Policy ID,

Microservices - Thrift, SOAP, REST

Microservices - Thrift, SOAP, REST

Synchronous REST = monolith

Synchronous REST is just a distributed monolith

REST code generation

Swagger focused code generation Swagger diff - are two API versions backward compatible Restlet - simple easy editor for swagger.

Commodity Computing

Webtask Sandboxing

API Transformer

API Transformer

Facebook Release Process

Release Process Web , Chuck Rossi, Facebook Release Engineering Mobile , Chuck Rossi, Facebook talk at Google 

Technology Radar

ThoughtWorks Technology Radar April 2016

Skype 12.04 fails on Ubuntu 14.04

Initially silently fails to start Run from terminal and it explains the error 32 bit lib fails to load libGl.so.1, add following to ~/.profile See  blog  entry sudo  /etc/ld.so.conf.d/skype.conf append /usr/lib32 export LD_PRELOAD="/usr/lib/i386-linux-gnu/mesa/libGL.so.1 $LD_PRELOAD" Then error mesg, see  blog  entry sudo update-alternatives --config i386-linux-gnu_gl_conf select 1 and enter sudo ldconfig

REST API Design tools

Swagger RAML Restlet  - simple easy editor for swagger.

Ubuntu 14.04 blank screen on Asus after boot

black-screen-with-nvidia-drivers-on-ubuntu

Tech Trends

W3Techs.com Browser Marketshare

Localization

localize.js

Documentation

MkDocs  using a  theme  provided by  Read the Docs , example Sphinx  documentation generator JavaDoc with UML via UMLGRAPH JavaDoc  with UML via yWorks

Microservice API tools

Kong - auth, logging, rate limit, etc. Swagger - REST API documentation, explore/test

SW Dev Envrionment

Spotify

Natty Java Natural Language Date Parser

Documentation Git repository

Samza, Kafka, Databus

LinkedIn Samza Stream processing Kafka messaging Databus - change capture