Friday, 8 December 2017

Sublime Text for Clojure

Sublime Text actually changed the text editor landscape. Before it was all TextMate which was stealing the thunder. But that is somehow a begone era. In this post we will have a sneak peak on using Sublime Text 3 for Clojure development. At this time there is no Emacs to cider kind of integration right from within Sublime Text, but it is possible to write one if required, however, connecting to nrepl from command line would just do fine. Also Clojure is not Java development where you would require to add breakpoint and step code execution as things are fairly straight forward to understand as opposed to having bazillion class files when it comes typical enterprise Java projects. However if one needs, you can resort to Eclipse with Counterclockwise plugin which works just fine.

1. Have Package Control installed.
2. In the beginning there were no parenthesis. It all started with a Big Bang, after which the amount of parenthesis started expanding. Since Clojure has the eternal brackets from the beginning of universe, we need a better bracket highlighter.
Add the following config to the Sublime user settings, so that the bracket highlighter works properly and does not conflict with the editor's own matching.
{
    "auto_match_enabled": false,
    "match_brackets": false,
    "match_brackets_angle": false,
    "match_brackets_braces": false,
    "match_brackets_content": false,
    "match_brackets_square": false,
    "match_tags": false
}
This package is pretty cool as it will show the matched brackets in the gutter.
3. To keep the sanity in own pattern matching of brackets and since most Lisp programmers are borderline hippie, to commemorate that let us use Rainbowth. Once installed add the following palette in the path ~/Library/Application Support/Sublime Text 3/Packages/User/Rainbowth.sublime-settings which is taken from Emacs's package to make the highlighting easier on eyes when used in conjunction with Solarized Dark theme.
{
  "palettes": {
    "default": ["#707183", "#7388d6", "#909183", "#709870", "#907373", "#6276ba", "#858580", "#80a880", "#887070"]
  }
}
But Rainbowth needs to update the colour scheme so we need to do some extraction of the desired scheme. Let's say we are using Solarized Dark theme, then do the following.
$ cd "/Applications/Sublime Text.app"
$ mkdir -p ~/temp/subl
# Solarized theme is inside the legacy package 
$ cp "Contents/MacOS/Packages/Color Scheme - Legacy.sublime-package" ~/temp/subl
# For some other themes which are not in the above, use
# $ cp "Contents/MacOS/Packages/Color Scheme - Default.sublime-package" ~/temp/subl
$ cd ~/temp/subl && unzip "Color Scheme - Legacy.sublime-package"
$ cp "Solarized (Dark).tmTheme" "~/Library/Application Support/Sublime Text 3/Packages/"
4. Add Clojure.sublime-settings in the same directory with the below contents so that by default, the source uses two spaces, which is the standard as per style guide and since we use `-` to separate words instead of camel case or `_`, the word separator helps us to easily select symbols with double click. Else, doing a double click on a word will select up to the separator.
{
    "tab_size": 2,
    "word_separators": "./\\()\"':,.;<>~!@#$%^&*|+=[]{}`~?"
}
That's all there is specific to Clojure development.

Theming
Next we can look into few enhancements and theming to make it easy on eyes. I generally prefer dark themes as it looks cool and tends towards Solarized Dark. While generally most articles say to use a lighter background with a darker text, it depends on various other conditions like the room lighting, and of course personal preference. Refer the Dark UI section of the post for citations.

1. Install Ayu colour scheme. We will be using Ayu Mirage with Solarized Dark theme. Add the following config under user settings.
{
    "color_scheme": "Packages/Solarized (Dark).tmTheme",
    "theme": "ayu-mirage.sublime-theme",
    "ui_big_tabs": false,
    "ui_fix_tab_labels": true,
    "ui_font_size_small": true,
    "ui_font_source_code_pro": true,
    "ui_separator": true,
    "ui_wide_scrollbars": true
}
2. Additional UI settings follows. Since I got used to Monaco font from early days of OS X, I tend to stick with it instead of newer ones like Menlo.
{
    "font_face": "Monaco",
    "font_options": [
        "subpixel_antialias"
    ],
    "font_size": 16,
    "gpu_window_buffer": true,
    "highlight_modified_tabs": true,
    "scroll_past_end": true,
    "show_full_path": true,
    "translate_tabs_to_spaces": true,
    "word_wrap": false
}

Other Enhancements
1. GitGutter to see source control changes.
2. SideBarEnhancements plugin which adds more options to the side bar right click menu.
3. EDN for Clojure EDN syntax highlighting.

Once done, the editor will look elegant as in the screenshot.



Dark UI
[1] Negative contrast computer displays (dark letters on a light background) generally provide a more legible image than positive contrast displays (light letters on a dark background). Either black characters on a white background or white characters on a black background have been found to be more visible than green, yellow-orange, blue or red characters.20

[2] The greatest difference among groups is on the black & white pair: while the majority of people without dyslexia prefer it (32.67%), only 13.64% of the participants with dyslexia chose black text on white background.
Participants without dyslexia tend to prefer color pairs with a higher color and brightness contrast while people with dyslexia read faster when color pairs have lower contrasts. For instance, the color pair which was the fastest to read by the participants with dyslexia was black & creme (mean of 0.214 for the fixation duration) while black text over yellow background presents the largest fixation duration mean (0.239 seconds). Although the pair off-black & off-white is the one recommended in Web design for dyslexics (Bradford, 2011), none of the users with dyslexia selected it. This pair presents the highest fixation duration mean for the participants without dyslexia (0.193 seconds).

[3] Maureen did mention two situations when a dark background makes sense.

However, if you are using a display in a dark environment, it’s better to use a dark background as it lets you keep your eyes dark adapted. That’s why controls for airplanes and GPS units for cars switch to a dark background at night. Usually, however, the results don’t look like symbols and text floating in the darkness of space…there’s still a sense of there being a dark surface to ground the view. So the concerns above are somewhat mitigated.

I’ve seen recommendations that white on black is better for aging eyes, and for people with low vision because it reduces the amount of light scattering and distorting the image.

[4] Chief among the reasons many of you have expressed for preferring dark backgrounds is the reduced strain placed on the eyes when staring at the screen for many hours. Many developers state that light text on a dark background is easier to read over longer periods of time than dark text on a light background.

Thursday, 7 December 2017

Integration Tests with pytest

pytest is a unit testing library and coupled with requests package can be used for writing integration tests. With integration tests, couple of things we would need is a way to distinguish between environments like local, staging, production, different datacenters and run tests in all or some specific tests per env and such.

1. We need a way to pass env during test runs. We will use pytest-variables for that which makes it easy to get variables passed in from command line available to the tests. Another way is to write conftest. So our requirements.txt file will now have pytest-variables[hjson]==1.5.1 apart from other packages.
2. We will name our pytest file starting which starts with test as something like test_api.py
3. Now on writing the tests. I would recommend that the python code be modular and well organised instead of spaghetti coding. Read Modular Programming with Python by Erik Westra if you are interested. We could separate parts into different file or have things organised under class and keep it under the same file. The point is separation of concerns, readability and maintainability (especially by other programmers). A thing with OOP is that go for composition as opposed to inheritance when possible.
4. We will go with how to write pytests in an object oriented fashion rather than just writing test_ functions. Let's call the below module as test_api.py
import pytest
import logging
import requests

__version__ = "0.0.42"  # corresponds to app version
logging.basicConfig(level=logging.DEBUG)
__timeout = 90  # seconds

def get_headers(props):
    """Return header map."""
    # ...

def http_post(params, data, headers=None):
    """Send an HTTP POST request."""
    return requests.post(params['server'] + params['endpoint'], data=data, headers=headers)

class FooAPI:
    """Sample Foo API helper class.""""

    def __init__:
        pass

@pytest.mark.timeout(__timeout)
class TestFooAPI:
    """Test Foo API service."""

    pytestmark = [pytest.mark.local, pytest.mark.ny1, pytest.mark.ams1, pytest.mark.ny1_green, pytest.mark.stg1]

    @pytest.fixture()
    def foo(self):
        """Return Foo object."""
        return FooAPI()

    def gen_name():
        """Standalone function that not pytest functions."""
        pass

    def test_add_user(self, variables, foo):
        """Test adding of user."""
        log = logging.getLogger('test_add_user')
        resp = foo.create_request(variables)
        headers = get_headers(variables)
        http_resp = http_post(variables, data, headers)
        assert http_resp.status_code == 200

    @pytest.mark.ams1_beta
    def test_beta_endpoint(self, variables, foo):
        """Test beta endpoint."""
        log = logging.getLogger("test_beta_endpoint")
        data = {'users': base64.b64encode('something'), 'bar': 'baz'}
        start = time.time()
        headers = get_headers(variables)
        log.info("\nheaders: %s", headers)
        http_resp = http_post(variables, data, headers)
        rtt = time.time() - start
        log.debug("\nrtt: %s\n", rtt)
        body = json.loads(http_resp.content)
        assert body['status'] is True
        assert body['code'] == 0
        assert http_resp.status_code == 200, 'Expecting 200 OK status code.'

class TestBarAPI:
    # ....

Any class that starts with Test and any function that starts with test will be executed by pytest by default. However these behaviour can be configured. Fixtures are variables that are passed to each test as argument before it runs. There are various pytest markers. If whole pytest functions within a pytest class needs to executed by many different environments, we can pass specify them as pytestmark = [pytest.mark.en1, pytest.mark.env2, ...]. The env1, env2 correspond to env config file name. And if we need to execute specific tests against specific env then, exclude them from the above list and annotate it above that particular function as @pytest.mark.ams1_beta, which means, if the env passed via command line is ams1_beta, only then this test will execute and since the env is not specified common in the pytestmark list, other tests will be skipped. Example usecase is running sanity tests (a subset of tests) against productions at a specific interval, where we do not have to run the full testsuite. The variables argument contains the config values defined in the config json which will be available to the test methods as a python dictionary.
5. The configuration files are under config directory. Example local.json is below. The name of the file corresponds to the env which we will be passing to the pytest via command line.
{
    "server": "http://localhost:8080",
    "endpoint": "/foo/test/",
    "key": "local_key.pem",
    "cert": "local_cert.pem",
    "env": "local",
    "json": true
}
6. Tying it all together using shell script. It is better to use virtualenv. So below script does a bit of housekeeping before running the tests.
#!/bin/bash

# Integration test with pytest.

TEST_DIR=test_api
VENV_DIR=venv
CONFIG_DIR=config

function activateVirtualEnv {
    echo "Activating venv .."
    if ! hash virtualenv 2>/dev/null; then
        echo "virtualenv not found. Installing .."
        pip install virtualenv
    fi
    virtualenv $VENV_DIR
    source $VENV_DIR/bin/activate
}

function installDeps {
    echo "Installing dependencies .."
    pip install -r ../requirements.txt
}

function prepareForTest {
    echo "Running integration test .."
    activateVirtualEnv
    installDeps
}

function deactivateVirtualEnv {
    echo "Deactivating venv .."
    deactivate
}

function displayhelp {
    echo "Usage: $0 options"
    echo "where options include:"
    echo "  local    run test in local env"
    echo "  ams1     run test in Amsterdam production datacenter 1"
    echo "  stg1     run test in staging enviroment 1"
}

# $1 result xml file name
# $2 env variable name
function run {
    echo "Running test against $2"
    # -n xdist flag
    # -s capture output
    # -v verbose
    # --variables The config file
    # -m marker
    py.test -n 4 -s -v --junitxml="$1" --variables "$CONFIG_DIR/$2.json" -m "$2" test_api.py
}

# $1 results.xml
# $2 env : local, rqa2, etc. corresponds to the filename in config
# To run against all machines in staging and production, use env as stg and prod respectively
# For integration test
# ./runtests.sh results.xml local
function main {
    result="$1"
    env="$2"
    echo "result: $result"
    echo "env: $env"
    echo "config: $CONFIG_DIR"

    cd $TEST_DIR
    prepareForTest
    if [[ $env == "stg" ]]; then
        run stg1.xml stg1
        run stg2.xml stg2
        python merge_junit_results.py stg1.xml stg2.xml > $result
    elif [[ $env == "prod" ]]; then
        run ny1.xml ny1
        run ams1.xml ams1
        python merge_junit_results.py ny1.xml ams2.xml > $result
    else
        # individual
        run $result $env
    fi
    return_code=$?
    deactivateVirtualEnv
}

main $@

exit $return_code
The merge_junit_results.py can be obtained from the cgoldber gist.
To run the test we run the command ./runtests.sh ${result}.xml ${env-config-file-name}, which will check for virtualenv and such and run the test. To run the tests in parallel use xdist.
7. Sample reqirements.text file would look like
pytest==3.0.7
pytest-variables[hjson]==1.5.1
pytest-xdist==1.15.0
requests==2.13.0
pytest-timeout==1.2.0
8. To integrate the results with jenkins, it needs to know the final result of the testsuite which will be present in the results.xml file. To parse that use the code below.
#!/bin/python

"""Extract test result failure, error values for Jenkins job."""

import xml.etree.ElementTree as ET
import sys


def parse_result(file):
    """Parse the given results.xml file."""
    root = ET.parse(file).getroot()
    if root.attrib["failures"] == "0" and root.attrib["errors"] == "0":
        print "ALLTESTSUCCESS"
    else:
        print "ERROR"
        sys.exit(1)


if __name__ == '__main__':
    print "junit filename: ", sys.argv[1]
    parse_result(sys.argv[1])

Now we have a full blown integration testsuite which can be integrated with CI/CD pipeline.

For clarity, sample folder structure will look like below.
test
├── requirements.txt
├── runtests.sh
└── test_api
    ├── __init__.py
    ├── local.xml
    ├── config
    │   ├── ams1.json
    │   ├── ams1_beta.json
    │   ├── local.json
    │   ├── local_beta.json
    │   ├── ny1.json
    │   ├── ny1_beta.json
    │   ├── ny1_green.json
    │   └── vm.json
    ├── merge_junit_results.py
    ├── parse_xml.py
    ├── result.xml
    ├── test_api.py
    └── venv

Wednesday, 6 December 2017

Passcode Lock iPhone even with TouchID Enabled

I found a way to force lock iPhone that has TouchID enabled. If done as below, then the phone requires passcode to unlock even when TouchID is configured to unlock the device.
1. Enable AssistiveTouch from Accessibility.
2. And under AssistiveTouch->Custom Actions, say for Long Press set it to SOS.

Now when we need to passphrase lock, long press the assistive touch button, which will dial the emergency SOS number. Cancel the call immediately and the phone gets locked. The only problem is that it makes the emergency sound when dialling.

Or second option is to press the unlock button and tap the wrong finger five times and the phone will lock itself. Whichever is easy works. Now how would the second option work with FaceID?

Tuesday, 5 December 2017

Erlang is Famous as it appeared on TV series

Did you know that Erlang is famous as in Hollywood famous, and you had not heard about it? Turns out a chat conversation in BlindSpot TV series 1 episode 6 says "Sybok: how quickly can you learn erlang?". And it indirectly implies that the TRAKZER app was written in Erlang. Anyway that is just a story. But yeah, how quickly can you learn Erlang dear?



But unfortunately, there is a twist to the story (spoiler alert). The girl (Ana Montes) at a later point in time uses python to break into the FBI system and then switches to perl. Which suggests that the above assumption is likely false or may be due to the lack of time and to give a clue to Patterson she used it. And the interesting part is the girl writes well documented code that follows PEP-257 at gunpoint. Now that is what I call software craftsmanship.

Monday, 4 December 2017

I Origins and Superstring Theory

So far I have restricted myself to not post anything that are not related to computers and technology in this blog, but I decided it is time to change that as I do not see why not to. And then this blog is not a lore on computing and my intentions change and my thoughts adapt to changing situations just like any normal human being. So with that said, let us take a small dive into the wilderness.

I am touched by the I Origins movie which I watched a couple of months ago and ever since I am wondering on and off about the existence of something unknown to me as in what we would call God. My thought process is like, organisms that do not have the PAX6 gene do not have any idea or perception about vision. And then the organism gets the gene added and it starts experiencing the world around it in a total different way. It might be able to handle the sensory perceptions or it might not.

Now what if there are forces, vibrations that we are not able to detect but exists all along with us. Superstring theory suggests that there are more dimensions than the ones we perceive. Kaluza-Klein gravity merges Einstein's general theory of relativity (gravity) with Maxwell's theory of electro-magnetism by adding a fifth dimension, but we do not observe them because they are compactified, which means are microscopic in nature. But there are other explanations that are not based on compactification, like projective geometry which says these extra dimensions are not physically real and thus cannot be observed. Or non-compactification theory which suggests that extra dimensions are physical, but relaxes the cylindricity condition (physics depending on the first four coordinates), but the effects are negligible when unit conversions are considered. Nevertheless the point is that there can be extra-dimensions that we do not really understand and perceive as we have not yet had that thought, or concrete evidence or we lack something analogous to the PAX6 gene that would allow us to perceive the world around us in a different way.

We do not have the answers yet, whatever that might be and the search continues, which makes this subject a very interesting fascination of mine.

A Song by Siri

I said something, Siri heard something else and in the end she sang a song. Not bad I would say.

Sunday, 3 December 2017

Solitude by Jehnny Beth

These are the words from the video Start Making Sense by Jehnny Beth on Beats 1. I like these words, the poetry.

I'm close to the burn out,
doing nothing is all I dream about.
Friendships, career, phone,
even when we're together we're all alone.
I wish I could freeze time,
make this morning last longer.
Everything I do on my own,
I remember better.
Songs to enjoy my solitude.

Tuesday, 28 November 2017

Dynamic Binary Analysis Using Hopper Disassembler on OS X

Dynamic code analysis in reverse engineering binaries is where we execute the binary for analysis, adding breakpoints, inspecting processor state and such. This is very useful to understand the program's algorithm and values much easier than static binary analysis. However when we analyse malware, running them is dangerous. But we can have an isolated machine where the program can be run and analysed remotely. This is an explanation of these techniques as well as a tutorial on how to reverse OS X crackme challenge (1-Sandwitch.zip | mirror).
This is a small app (by HAWKE, 2009) which requires us to find the correct serial key. The goal is to produce a keygen as opposed to patching. Patching is just easy as changing one instruction.

1. Load the Sandwich.app in Hopper Disassembler, choose the default and proceed to disassembly screen.
2. Instead of remote server, we will connect to the current system using Hopper Debugger Server. Download, install and run the app which will launch the debug server locally.

3. Now in the Hopper, navigate to Debug->Select Debugger... and the server will be listed under local servers section. Double click to launch the server window.
4. The path will be auto filled to something like /Users/.../Sandwich.app/Contents/MacOS/Sandwich. Click the Play icon under controls which will run the Sandwich.app under the debugger.
5. Check the left pane for the available methods. We can see -[SandwichAppDelegate validateSerial:] method. Check the pseudo code to get an idea of the algorithm.

6. Now is the time to add breakpoints so that we can step and check register values as the program runs. So add to the initial address 00001b2d 55 push ebp under that section.
7. Go back to the running program and enter some serial number and click validate, which will make the debugger stop at the breakpoint.
8. The first conditional check is cmp eax, 0x13 which checks if the serial number is 19 decimals long.

9. Second check is mov dword [esp+0x28+var_20], 0x2034 ; @"-" which then checks the number of components separated by - and compares it with cmp eax, 0x4, which means, the serial key has four dashes in it. So the format is xxxx-xxxx-xxxx-xxxxx

10. If all the above passes, it gets each block (chunk separated by dash) and check if length is four.

11. Next it takes the first block 0x0 gets the int value and place it into esi register mov esi, eax.

12. Then it takes the second block 0x1, get the int value and adds it to the block 1 esi value from above step and place it into esi register itself.

13. There is no check for third block which means it can be any random characters of length four.
14. The last block is constructed by computing shift arithmetic right by two of the current esi sar esi, 0x2 from step 12 and then subtracting it from 0x19c5 the HEX for 6597 loaded into edx and then comparing the obtained value to the last block from the serial key loaded into eax

15. If it match, the serial key is correct and a success dialog will be shown. Now that we know the algorithm, we can write the keygen.


The keygen code
#!/usr/bin/python

"""Sandwich OS X crackme keygen by kadaj.

App: https://reverse.put.as/wp-content/uploads/2010/05/1-Sandwich.zip
Tue 28 Nov 17
"""

import random

class SandwichKeygen:
    """Sandwich key generator."""

    __block_3_mn = 6597  # block 3 magic number

    def _gen_random(self):
        """Return random number in range of 4 digits."""
        return random.randrange(1000, 9999)

    def _get_block(self):
        """Get a block containing 4 digits."""
        return str(self._gen_random())

    def _get_block_3(self, block_1, block_2):
        """Generate the last block from block 1."""
        sar_2 = (int(block_1) + int(block_2)) >> 2
        return str(self.__block_3_mn - sar_2)

    def generate(self):
        """Generates the key."""
        block_0 = self._get_block()
        block_1 = self._get_block()
        return '-'.join([block_0, block_1, str(self._gen_random()), self._get_block_3(block_0, block_1)])

if __name__ == '__main__':
    keygen = SandwichKeygen()
    print keygen.generate()

# Few keys
1111-2222-0000-5764
2681-8944-2259-3691
6773-1497-8343-4530
5903-2519-1731-4492
9613-1094-3343-3921

Tuesday, 14 November 2017

Cherry Blossoms with CherryPy

CherryPy is a python web framework which I really like. It is minimalistic in nature, is very pleasant to work with, and still we can accomplish much. Below I will detail a basic web app setup with CherryPy demonstrating basic usage which one can expand further.
# blossoms.py
"""A blossoms server with CherryPy."""

import cherrypy
import os

class CherryBlossom(object):
    """The server route handler methods."""

    def __init__(self):
        pass

    @cherrypy.expose
    def index(self):
        """The root url, when not mapped to static response, this action will be invoked."""
        return self.about()

    @cherrypy.expose
    def about(self):
        return 'Enjoy cherry blossoms.'

    @cherrypy.expose(['greet'])
    @cherrypy.tools.json_out()
    def get_greeting_message(self, name):
        """Return greetings to the given name in JSON format."""
        return {'msg': 'Greetings %s' % name'}

    @cherrypy.tools.register('before_finalize', priority=60)
    def secureheaders():
        """Update server response headers."""
        headers = cherrypy.response.headers
        headers['X-Frame-Options'] = 'DENY'
        headers['X-XSS-Protection'] = '1; mode=block'
        headers['X-Powered-By'] = 'Cherry Blossoms'
        headers['Server'] = 'Sakura'

class Blossoms(object):
    """Server configuration and initialisation."""

    def __init__(self):
        pass

    def start(self):
        """Start the service."""
        blossom = CherryBlossom()
        # routes
        dispatcher = cherrypy.dispatch.RoutesDispatcher()
        dispatcher.explicit = False
        dispatcher.connect('home', '/', blossom.index)
        dispatcher.connect('about', '/about/', controller=blossom, action='about', conditions=dict(method=['GET']))
        dispatcher.connect('greetings', '/greet/{name}/', controller=blossom, action='greet', conditions=dict(method=['GET']))
        # config
        root_dir = os.path.dirname(os.path.abspath(__file__))
        global_conf = root_dir + '/blossoms.conf'
        cherrypy.config.update(global_conf)
        conf = {
            '/': {
                'request.dispatch' : dispatcher,
                'tools.staticdir.dir': root_dir
            }
        }
        app = cherrypy.tree.mount(None, '/', global_conf)
        app.merge(conf)
        if hasattr(cherrypy.engine, "signal_handler"):
            cherrypy.engine.signal_handler.subscribe()
        if hasattr(cherrypy.engine, "console_control_handler"):
            cherrypy.engine.console_control_handler.subscribe()
        cherrypy.engine.start()
        cherrypy.engine.block()
# pyserver.py
from blossoms import Blossoms

blossom = Blossoms()
blossom.start()
The configuration are present in the blossoms.conf file.
[global]
server.socket_host: '127.0.0.1'
server.socket_port: 1729
server.thread_pool: 8

[/]
tools.gzip.on: True
tools.staticdir.on: True
tools.staticdir.index: 'index.html'
tools.staticdir.debug: True,
tools.secureheaders.on = True,
tools.response_headers.on: True,
tools.response.stream: True,
tools.sessions.on = True
tools.sessions.secure = True
tools.sessions.httponly = True
log.screen: True
Calling the /, FQDN alone is configured to serve static content. The GET endpoints are /greet/{name} and /about/. The RoutesDispatcher uses routes package to do the dispatch. The requirements.txt is below.
cherrypy==11.1.0
routes==2.4.1
We load the config and merge it with few updated setting from the code before starting the server. We have a server that handles static content, does REST based routing, custom response headers, JSON, html responses, with configurable settings. Enjoy cherry blossoms.

Saturday, 11 November 2017

Erlang on Android with Termux

Termux is a terminal emulator and Linux environment app for Android. This app, I would say is one of the best things ever happened on an Android platform. It helps us to install packages as in Maemo. The erlang package is available and can be installed using pkg install erlang. It currently installs Erlang/OTP 20, pretty nice.

Wednesday, 8 November 2017

Accessing private functions in Clojure

Nothing in Clojure is truly private. The defn- is more of a way to convey that it is private to the readers. We can access private method by using var quote syntax.
The var special form or the #' reader macro (see Reader) can be used to get an interned Var object instead of its current value.
user=> (ns private)
nil
private=> (defn- secret[] "oops")
#'private/secret

private=> (ns user)
nil
user=> (private/secret)
CompilerException java.lang.IllegalStateException: var: #'private/secret is not public, compiling:(NO_SOURCE_PATH:4:1) 

user=> (#'private/secret)
"oops"

Monday, 6 November 2017

Functional Conf 2017

Functional Conf 2017 is happening in Bangalore this year, which is also hosting the first Erlang & Elixir Factory Lite Conference in India. Also there are separate workshops we can register.

I am attending Reactive Systems & MicroServices Architecture by Francesco Cesarini, along with all Functional Conf sessions with all access pass, and Deep Dive into Erlang Ecosystem by Robert Virding.

Update: The workshops and discussions were so very amazing.

Sunday, 5 November 2017

Monkey Patching with Clojure intern Function

The intern function in Clojure is a powerful construct. This is similar to MOP in Groovy. It really comes in handy in many situations. I particularly use it in unit tests to mock (provide override) implementation of data layer methods and to monkey patch third party libraries dynamically.
; com.external.core.clj
(ns com.external.core)

(defn tangle 
  "An internal method which takes an argument but always return the tangle string"
  [msg]
  "tangle")
Now let us say that this is a third party library that we included from maven (or clojars) as project dependency. And we found an issue, the issue being it should return "untangle" instead of "tangle" but we do not have the luxury of upgrading the library to the latest version which has the fix. This is where intern comes in handy.
; com.example.init.clj
(ns com.example.init)

(defn untangle
  "Function that fixes a bug in an function that lives in an external namespace"
  [x]
  "untangle")

;; Update the binding, means all callers will see the new patched function, even callers in other any libraries
(intern 'com.external.core 'tangle untangle)
What happens here is that the intern updates the binding of function tangle in the namespace com.external.core to the untangle function defined in com.example.init namespace. From now on any call to the original function will use the updated patched function instead. This is very neat in certain situations where we need to monkey patch. This is not function override, because this change is global as in the entire app sees the new definition. Note that the function signature must be the same. Else it will break the callers.

This is also helpful in unit testing Clojure code. Say we have a service layer and calls into data layer. But we do not want to hit the DB or API methods, as we are not really performing integration tests. So we can provide mock methods inside the deftest with mock results, which will easily help in testing our service layer code.

There is a caveat. This does not work well with Clojure macros, as macros are compile time which generates code. So at runtime, there really is no macro code binding that we can override.

Thursday, 26 October 2017

Use binding as an alternative to passing state around in Clojure

Let's say we have already existing functions (referred to as inner functions) called by another function (outer function) and the inner functions needs some state data so that when exception occurs, that data can be logged. However, this requires passing state around to all these inner functions and modifying the method signature. Which is fine, but depending on the taste, it might not look elegant. There is another way where the state can be bound using binding and does not have to be passed explicitly by the caller. Example follows.
(declare ^:dynamic state)

(defn err []
  ; does many other things
  (println state)
  ; .. return response
  )

(defn process []
  (binding [state 1] 
    (err)  ; 1
    (set! state 2)  ; update value of state
    (err)))  ; 2

(process)

; output
; 1
; 2
; nil
This however is an anti-pattern as the state var is being implicit to the function.

Wednesday, 25 October 2017

Unbind Active Directory from OS X

When your enterprise uses Active Directory for identity management, the OS X user account by default will be bound to the AD service. The convenience it brings is that we can change the AD account password by changing the OS X user account password and all services like WiFi, printer authentication will be updated as it will update the keychain as well. If the OS X user account is not bounded, we will have to manually remove/update the necessary cached service password from keychain each time. However, it is problem that the AD will get information on local OS X account activity and will get synced when connected to the internal network. You cannot be a ghost then. Now comes the unbinding part.
1. Check current AD settings
sudo dsconfigad -show
If bounded, it will print something like,
Active Directory Domain          = stalker.weird.org
Computer Account                 = xxx

Advanced Options - User Experience
  Create mobile account at login = Enabled|Disabled
     Require confirmation        = Enabled|Disabled
  Force home to startup disk     = Enabled|Disabled
     Mount home as sharepoint    = Enabled|Disabled
  Use Windows UNC path for home  = Enabled|Disabled
     Network protocol to be used = smb
  Default user Shell             = /bin/bash

Advanced Options - Mappings
  Mapping UID to attribute       = xxx
  Mapping user GID to attribute  = xxx
  Mapping group GID to attribute = xxx
  Generate Kerberos authority    = Enabled|Disabled

Advanced Options - Administrative
  Preferred Domain controller    = xxx
  Allowed admin groups           = xxx
  Authentication from any domain = Enabled|Disabled
  Packet signing                 = xxx
  Packet encryption              = xxx
  Password change interval       = 000
  Restrict Dynamic DNS updates   = xxx
  Namespace mode                 = xxx
2. Install Enterprise Connect app from Apple that will help us to easily reconnect and change AD password. The org IT must support it. You won't find it in the App store.
3. Unbind AD.
dsconfigad -f -r -u <ad_username>
This above command will unbind the OS X user account from the AD server. Check if AD service is active by running the command in step 1 again. It should print nothing.
4. Next if we need to bind to AD, use Enterprise Connect -> Reconnect -> Change Password. Which is easy way. Else we can use dsconfigad utility to do the binding.

Enjoy stealth mode.

Monday, 23 October 2017

Data Flow Pipeline in Clojure

Error handling in Clojure requires some serious thought. There are many ways. Some gets the argument that it looks less functional, especially using exceptions as the basic mechanism to handle errors since it give a Java feel to the code. Monads and short circuiting (either monad, error monad, etc.) is also popular among functional programming practitioners. However, I hate monads with passion, apparently for no reason. Well, because it is a workaround to the purely typed functional programming languages like Haskell to handle state, side effects which they brought upon themselves. Clojure being Lisp and Lisp being awesome, we really does not require all these complications. I have been thinking about core.async channels as a way to build data flow pipeline and handle errors, but it does not map very well, when in comparison with Erlang, its processes, supervisors and such. Modelling Erlang kind of systems on JVM without green threads and non-os scheduler is super hard, and less performant. It's all in the VM.

Now to the point. There is a very simple way to build data flow pipeline and handle errors by short circuiting and still keeping it elegant. The build blocks are reduce and reduction. And specifically, it is the reduction function that helps to break the recursion if error occurs. Example code follows.
;; pipeline.clj

#!/usr/bin/env boot

(defn deps [new-deps]
  (merge-env! :dependencies new-deps))

(deps '[[org.clojure/clojure "1.8.0"]])

(defn validate-true [x]
  (println "validate-true")
  {:status true :state {:id 123 :foo "bar"}})

(defn validate-false [x]
  (println "validate-false")
  {:status false :code 1 :msg "anti gravity"})

(defn validate-false-1 [x]
  (println "validate-false")
  {:status false :code 1 :msg "heisenbug"})

(defn pipeline [fns]
  (reduce (fn [x fun]
    (if (:status x)
      (fun (:state x))
      (reduced x)))
    {:status true}
    fns))

(println "pipeline 0")
(println (pipeline [validate-false validate-true validate-false validate-true validate-true]))

(println "\npipeline 1")
(println (pipeline [validate-true validate-false-1 validate-true validate-false validate-true]))

(println "\npipeline 2")
(println (pipeline [validate-true validate-true validate-false validate-true validate-true]))

(println "\npipeline")
(println (pipeline [validate-true validate-true validate-true]))
# output
pipeline 0
validate-false
{:status false, :code 1, :msg anti gravity}

pipeline 1
validate-true
validate-false
{:status false, :code 1, :msg heisenbug}

pipeline 2
validate-true
validate-true
validate-false
{:status false, :code 1, :msg anti gravity}

pipeline
validate-true
validate-true
validate-true
{:status true}
# boot.properties
#https://github.com/boot-clj/boot
BOOT_CLOJURE_NAME=org.clojure/clojure
BOOT_VERSION=2.7.1
BOOT_CLOJURE_VERSION=1.8.0
The pipeline accepts a list of functions and chains them together. Each reduction can pass in the required arguments necessary for the next function in the chain. These functions must be composable to make this work. The reduced function checks for a recursion termination condition apart from the reduction termination. When the condition which is the {:status false ..} is met, it short circuits the evaluation and exits the recursion, returning the value of the computation. This value can then be matched and responded accordingly. In case of web app, a suitable ring response can be generated. Simple!

The pipeline can be modified to accept a collection of [ok-fn err-fn] so that instead of having a generic one error handler, we can have two different paths and localised handling of errors.
(defn pipeline [fns]
  (reduce (fn [x fun]
    (if (:status x)
      (if (= (type fun) clojure.lang.PersistentVector) (((first fun) (:state x))) (fun (:state x)))
      (if (= (type fun) clojure.lang.PersistentVector) (reduced ((second fun) x)) (reduced x))))
    {:status true}
    fns))
(defn validate-false-s1 [x]
  (println "special error handling fun in pipeline")
  {:status false :code -1 :msg "nop"}) 

(println "\npipeline separate flows")
(println (pipeline [validate-true validate-false [validate-true validate-false-s1] validate-true]))
; output
pipeline separate flows
validate-true
validate-false
special error handling fun in pipeline
{:status false, :code -1, :msg nop}

The pipeline is very flexible and can be modified to suite variety of use cases. The contract does not always have to be a map with :status. We can use core.match to do various types of conditional branching.

Update: Using try..catch exceptions are fine. It does not make code any less functional as long as you return values. Plus using functional language does not mean that whole code will be functional. There will be a few parts that are less functional, and that is totally fine. The point here is to be pragmatic. The main use of monad is to sequence code.

Thursday, 19 October 2017

Lock BlackBerry KEYone With Password When Using Fingerprint

BlackBerry KEYone, an advancement from BlackBerry Passport, which however uses Android, an OS not so great when compared to QNX, but still fine given the fact that it can be hardened and there are apps that makes business get done. Getting required apps on BlackBerry 10 is really a pain, no matter how great the device and OS is. Plus developers like me can program KEYone using any of the supported JVM languages instead of banging my head against C/C++, a pain for any functional programmer accustomed to living in the JVM world.

So, now to the point. BlackBerry KEYone has fingerprint sensor on the spacebar. However, we know fingerprint is a security issue. When confiscated, we are legally required to unlock the phone if the finger is still on the hand. But, that is not the case with passwords. There are a lot of defences against providing authorities with your password. So KEYone can lock the device which will then require you to enter the password even if it is configured to unlock using fingerprint. For that, long press the key K, which opens BlackBerry Launcher and initially it asks whether you need to enable the app to administer the device. Allow access and the device will be locked using that keyboard shortcut, which then requires the password to unlock. Rest of the time, unlock using fingerprint. Makes life easier without compromising security. BlackBerry rocks!

Tuesday, 10 October 2017

jamf equals no privacy

jamf is used for managing Apple devices in the enterprise. However it is a nasty little piece of software. It always sends online, offline status to enterprise JSS endpoint. Few excerpts from /var/log/jamf.log.
Tue Oct 03 15:26:03 purgatory jamf[52]: Daemon starting
Tue Oct 03 15:26:05 purgatory jamf[363]: 
There was an error.

     Connection failure: "The Internet connection appears to be offline."

Tue Oct 03 15:26:06 purgatory jamf[52]: Network state changed, checking for policies...
Tue Oct 03 15:26:06 purgatory jamf[407]: Could not connect to the JSS. Looking for cached policies...
Tue Oct 03 15:26:07 purgatory jamf[52]: Network state changed, checking for policies...
Tue Oct 03 15:26:10 purgatory jamf[52]: Network state changed, checking for policies...
Tue Oct 03 15:26:13 purgatory jamf[52]: Network state changed, checking for policies...
Tue Oct 03 15:26:29 purgatory jamf[52]: Informing the JSS about login for user castiel
Tue Oct 03 15:26:38 purgatory jamf[52]: Network state changed, checking for policies...
Tue Oct 03 15:27:12 purgatory jamf[52]: Network state changed, checking for policies...
Tue Oct 03 15:27:17 purgatory jamf[52]: Network state changed, checking for policies...
Tue Oct 03 15:27:38 purgatory jamf[52]: Network state changed, checking for policies...
Tue Oct 03 15:27:56 purgatory jamf[52]: Network state changed, checking for policies...
Tue Oct 03 15:28:03 purgatory jamf[52]: Network state changed, checking for policies...
Tue Oct 03 15:28:04 purgatory jamf[2573]: Checking for policies triggered by "networkStateChange" for user "castiel"...
Tue Oct 03 15:28:04 purgatory jamf[2356]: Checking for policies triggered by "networkStateChange" for user "castiel"...
Tue Oct 03 15:28:04 purgatory jamf[2087]: Checking for policies triggered by "networkStateChange" for user "castiel"...
Tue Oct 03 15:28:07 purgatory jamf[2573]: Could not connect to the JSS. Looking for cached policies...
Tue Oct 03 15:28:07 purgatory jamf[2087]: Could not connect to the JSS. Looking for cached policies...
***
Wed Oct 04 20:02:06 purgatory jamf[13815]: Checking for policies triggered by "recurring check-in" for user "castiel"...
Wed Oct 04 20:02:10 purgatory jamf[13815]: Could not connect to the JSS. Looking for cached policies...
Wed Oct 04 20:05:33 purgatory jamf[52]: Network state changed, checking for policies...
***
Thu Oct 05 09:26:25 purgatory jamf[99672]: Checking for policies triggered by "networkStateChange"...
Thu Oct 05 10:04:40 purgatory jamf[52]: Informing the JSS about login for user root
***
Sat Oct 07 07:51:30 purgatory jamf[64843]: Checking for policies triggered by "networkStateChange" for user "castiel"...
Sat Oct 07 08:02:52 purgatory jamf[69647]: Checking for policies triggered by "recurring check-in" for user "castiel"...
Sat Oct 07 08:02:55 purgatory jamf[69647]: Executing Policy Enable local firewall
Sat Oct 07 08:02:56 purgatory jamf[69647]: Executing Policy Inventory Daily
Sat Oct 07 08:02:57 purgatory jamf[69647]: Executing Policy Update Username Field in Inventory
Sat Oct 07 08:29:25 purgatory jamf[52]: Network state changed, checking for policies...
Sat Oct 07 08:30:27 purgatory jamf[87635]: Checking for policies triggered by "networkStateChange" for user "castiel"...
Sat Oct 07 08:30:28 purgatory jamf[87262]: Checking for policies triggered by "recurring check-in" for user "castiel"...
Sat Oct 07 08:31:45 purgatory jamf[87635]: Could not connect to the JSS. Looking for cached policies...
Sat Oct 07 08:31:46 purgatory jamf[87262]: Could not connect to the JSS. Looking for cached policies...
Sat Oct 07 08:31:46 purgatory jamf[87262]: Executing Offline Policy Enable local firewall
Sat Oct 07 08:49:57 purgatory jamf[97458]: Checking for policies triggered by "recurring check-in" for user "castiel"...
Sat Oct 07 08:51:15 purgatory jamf[97458]: Could not connect to the JSS. Looking for cached policies...
Sat Oct 07 08:51:15 purgatory jamf[97458]: Executing Offline Policy Enable local firewall
Sat Oct 07 09:09:24 purgatory jamf[8836]: Checking for policies triggered by "recurring check-in" for user "castiel"...
Sat Oct 07 09:10:41 purgatory jamf[8836]: Could not connect to the JSS. Looking for cached policies...
Sat Oct 07 09:10:41 purgatory jamf[8836]: Executing Offline Policy Enable local firewall
***
Mon Oct 09 10:27:46 purgatory jamf[20026]: Checking for policies triggered by "recurring check-in" for user "castiel"...
Mon Oct 09 13:45:26 purgatory jamf[28409]: Checking for policies triggered by "recurring check-in" for user "castiel"...
Mon Oct 09 13:45:29 purgatory jamf[28409]: Executing Policy Inventory Daily
Mon Oct 09 14:16:56 purgatory jamf[28409]: Error running recon: Connection failure: "The request timed out."
Mon Oct 09 14:20:26 purgatory jamf[52]: Daemon shutdown completed
Mon Oct 09 14:20:26 purgatory jamf[52]: Daemon exiting
So every time the jamf infected computer goes online or offline, changed to root etc., the Sauron will be notified about it. More interesting part comes next.

It tracks all applications used by the users and the amount of time spend with it. That treasure is in /Library/Application Support/JAMF/Usage folder. There will be folders like 2017-10-07, 2017-10-08, 2017-10-09. Looking at one of those folders will give logs like (null).plist, idle.plist, castiel.plist, etc.

Let us see what castiel.plist has to offer.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>/Applications/Calendar.app</key>
    <dict>
        <key>foremost</key>
        <string>0</string>
        <key>open</key>
        <string>968</string>
        <key>secondsforemost</key>
        <string>0</string>
        <key>secondsopen</key>
        <string>58112</string>
        <key>version</key>
        <string>9.0</string>
    </dict>
    <key>/Applications/Cisco/Cisco AnyConnect Secure Mobility Client.app</key>
    <dict>
        <key>foremost</key>
        <string>1</string>
        <key>open</key>
        <string>1911</string>
        <key>secondsforemost</key>
        <string>55</string>
        <key>secondsopen</key>
        <string>114686</string>
        <key>version</key>
        <string>4.1.08005</string>
    </dict>
    <key>/Applications/GIMP.app</key>
    <dict>
        <key>foremost</key>
        <string>0</string>
        <key>open</key>
        <string>1911</string>
        <key>secondsforemost</key>
        <string>0</string>
        <key>secondsopen</key>
        <string>114685</string>
        <key>version</key>
        <string>2.8.18</string>
    </dict>
    <key>/Applications/GitHub Desktop.app</key>
    <dict>
        <key>foremost</key>
        <string>0</string>
        <key>open</key>
        <string>1911</string>
        <key>secondsforemost</key>
        <string>0</string>
        <key>secondsopen</key>
        <string>114685</string>
        <key>version</key>
        <string>Hasty Things Done Hastily</string>
    </dict>
    <key>/Applications/Google Chrome.app</key>
    <dict>
        <key>foremost</key>
        <string>0</string>
        <key>open</key>
        <string>1911</string>
        <key>secondsforemost</key>
        <string>0</string>
        <key>secondsopen</key>
        <string>114685</string>
        <key>version</key>
        <string>63.0.3223.8</string>
    </dict>
    <key>/Applications/Mail.app</key>
    <dict>
        <key>foremost</key>
        <string>2</string>
        <key>open</key>
        <string>1911</string>
        <key>secondsforemost</key>
        <string>134</string>
        <key>secondsopen</key>
        <string>114686</string>
        <key>version</key>
        <string>10.3</string>
    </dict>
    <key>/Applications/Notes.app</key>
    <dict>
        <key>foremost</key>
        <string>1</string>
        <key>open</key>
        <string>1911</string>
        <key>secondsforemost</key>
        <string>27</string>
        <key>secondsopen</key>
        <string>114686</string>
        <key>version</key>
        <string>4.4</string>
    </dict>
    <key>/Applications/Photos.app</key>
    <dict>
        <key>foremost</key>
        <string>0</string>
        <key>open</key>
        <string>1911</string>
        <key>secondsforemost</key>
        <string>0</string>
        <key>secondsopen</key>
        <string>114685</string>
        <key>version</key>
        <string>2.0</string>
    </dict>
    <key>/Applications/Postman.app</key>
    <dict>
        <key>foremost</key>
        <string>0</string>
        <key>open</key>
        <string>1911</string>
        <key>secondsforemost</key>
        <string>0</string>
        <key>secondsopen</key>
        <string>114685</string>
        <key>version</key>
        <string>5.2.1</string>
    </dict>
    <key>/Applications/Preview.app</key>
    <dict>
        <key>foremost</key>
        <string>0</string>
        <key>open</key>
        <string>1911</string>
        <key>secondsforemost</key>
        <string>0</string>
        <key>secondsopen</key>
        <string>114686</string>
        <key>version</key>
        <string>9.0</string>
    </dict>
    <key>/Applications/Reminders.app</key>
    <dict>
        <key>foremost</key>
        <string>1</string>
        <key>open</key>
        <string>54</string>
        <key>secondsforemost</key>
        <string>31</string>
        <key>secondsopen</key>
        <string>3272</string>
        <key>version</key>
        <string>4.0</string>
    </dict>
    <key>/Applications/Safari Technology Preview.app</key>
    <dict>
        <key>foremost</key>
        <string>65</string>
        <key>open</key>
        <string>1911</string>
        <key>secondsforemost</key>
        <string>3920</string>
        <key>secondsopen</key>
        <string>114685</string>
        <key>version</key>
        <string>11.1</string>
    </dict>
    <key>/Applications/Slack.app</key>
    <dict>
        <key>foremost</key>
        <string>56</string>
        <key>open</key>
        <string>1892</string>
        <key>secondsforemost</key>
        <string>3381</string>
        <key>secondsopen</key>
        <string>113558</string>
        <key>version</key>
        <string>2.8.1</string>
    </dict>
    <key>/Applications/Sublime Text.app</key>
    <dict>
        <key>foremost</key>
        <string>33</string>
        <key>open</key>
        <string>1662</string>
        <key>secondsforemost</key>
        <string>2026</string>
        <key>secondsopen</key>
        <string>99735</string>
        <key>version</key>
        <string>Build 3143</string>
    </dict>
    <key>/Applications/Utilities/Keychain Access.app</key>
    <dict>
        <key>foremost</key>
        <string>0</string>
        <key>open</key>
        <string>1911</string>
        <key>secondsforemost</key>
        <string>0</string>
        <key>secondsopen</key>
        <string>114685</string>
        <key>version</key>
        <string>9.0</string>
    </dict>
    <key>/Applications/Utilities/Terminal.app</key>
    <dict>
        <key>foremost</key>
        <string>1</string>
        <key>open</key>
        <string>1908</string>
        <key>secondsforemost</key>
        <string>9</string>
        <key>secondsopen</key>
        <string>114527</string>
        <key>version</key>
        <string>2.7.3</string>
    </dict>
    <key>/Applications/VLC.app</key>
    <dict>
        <key>foremost</key>
        <string>0</string>
        <key>open</key>
        <string>1911</string>
        <key>secondsforemost</key>
        <string>0</string>
        <key>secondsopen</key>
        <string>114686</string>
        <key>version</key>
        <string>2.2.6</string>
    </dict>
    <key>/Applications/[...snip..]Crypt.app</key>
    <dict>
        <key>foremost</key>
        <string>0</string>
        <key>open</key>
        <string>1911</string>
        <key>secondsforemost</key>
        <string>0</string>
        <key>secondsopen</key>
        <string>114685</string>
        <key>version</key>
        <string>1.xx</string>
    </dict>
    <key>/Applications/Xcode.app</key>
    <dict>
        <key>foremost</key>
        <string>0</string>
        <key>open</key>
        <string>1911</string>
        <key>secondsforemost</key>
        <string>0</string>
        <key>secondsopen</key>
        <string>114686</string>
        <key>version</key>
        <string>9.0</string>
    </dict>
    <key>/Applications/iTunes.app</key>
    <dict>
        <key>foremost</key>
        <string>0</string>
        <key>open</key>
        <string>1911</string>
        <key>secondsforemost</key>
        <string>0</string>
        <key>secondsopen</key>
        <string>114686</string>
        <key>version</key>
        <string>12.7</string>
    </dict>
    <key>/System/Library/CoreServices/CoreServicesUIAgent.app</key>
    <dict>
        <key>foremost</key>
        <string>0</string>
        <key>open</key>
        <string>1911</string>
        <key>secondsforemost</key>
        <string>0</string>
        <key>secondsopen</key>
        <string>114685</string>
        <key>version</key>
        <string>168.3</string>
    </dict>
    <key>/System/Library/CoreServices/Finder.app</key>
    <dict>
        <key>foremost</key>
        <string>1</string>
        <key>open</key>
        <string>1911</string>
        <key>secondsforemost</key>
        <string>55</string>
        <key>secondsopen</key>
        <string>114685</string>
        <key>version</key>
        <string>10.12.5</string>
    </dict>
    <key>/System/Library/CoreServices/SystemUIServer.app</key>
    <dict>
        <key>foremost</key>
        <string>0</string>
        <key>open</key>
        <string>1911</string>
        <key>secondsforemost</key>
        <string>0</string>
        <key>secondsopen</key>
        <string>114686</string>
        <key>version</key>
        <string>1.7</string>
    </dict>
    <key>/System/Library/CoreServices/UserNotificationCenter.app</key>
    <dict>
        <key>foremost</key>
        <string>0</string>
        <key>open</key>
        <string>1911</string>
        <key>secondsforemost</key>
        <string>0</string>
        <key>secondsopen</key>
        <string>114686</string>
        <key>version</key>
        <string>3.3.0</string>
    </dict>
    <key>/System/Library/CoreServices/loginwindow.app</key>
    <dict>
        <key>foremost</key>
        <string>1685</string>
        <key>open</key>
        <string>1911</string>
        <key>secondsforemost</key>
        <string>101144</string>
        <key>secondsopen</key>
        <string>114686</string>
        <key>version</key>
        <string>9.0</string>
    </dict>
    <key>/System/Library/Frameworks/ScreenSaver.framework/Resources/ScreenSaverEngine.app</key>
    <dict>
        <key>foremost</key>
        <string>0</string>
        <key>open</key>
        <string>1911</string>
        <key>secondsforemost</key>
        <string>0</string>
        <key>secondsopen</key>
        <string>114686</string>
        <key>version</key>
        <string>5.0</string>
    </dict>
    <key>/System/Library/Frameworks/Security.framework/Versions/A/MachServices/SecurityAgent.bundle</key>
    <dict>
        <key>foremost</key>
        <string>0</string>
        <key>open</key>
        <string>1911</string>
        <key>secondsforemost</key>
        <string>0</string>
        <key>secondsopen</key>
        <string>114686</string>
        <key>version</key>
        <string>9.0</string>
    </dict>
    <key>/System/Library/Frameworks/WebKit.framework/Versions/A/XPCServices/com.apple.WebKit.WebContent.xpc</key>
    <dict>
        <key>foremost</key>
        <string>0</string>
        <key>open</key>
        <string>1911</string>
        <key>secondsforemost</key>
        <string>0</string>
        <key>secondsopen</key>
        <string>114686</string>
        <key>version</key>
        <string>12603</string>
    </dict>
    <key>/private/var/folders/9m/_fh0czw947g8q7pdhxbfbjdh0000gn/T/AppTranslocation/AB999B43-95BD-4B9E-880D-6C59DFD81558/d/Base64.app</key>
    <dict>
        <key>foremost</key>
        <string>1</string>
        <key>open</key>
        <string>72</string>
        <key>secondsforemost</key>
        <string>7</string>
        <key>secondsopen</key>
        <string>4344</string>
        <key>version</key>
        <string>1.0</string>
    </dict>
</dict>
</plist>
Oops, now the pointy honchos knows which applications I am using and for how long in a day. New age micro-management.

The cure
The cure is very simple however. Turn off the jamf service. And next time when we want to turn it on, say to see if IT has pushed some clever software, clear all the logs before hand, close all apps. Then load the daemon back.
sudo launchctl load /Library/LaunchDaemons/com.jamfsoftware.jamf.daemon.plist 
sudo launchctl load /Library/LaunchDaemons/com.jamfsoftware.task.1.plist #com.jamfsoftware.task.{n}.plist check the folder for correct number
Let the update get pushed to the system, then turn it off.
Also as per policy, we cannot set the OS X firewall in stealth mode, blocking all connections. It automatically changes to "on" mode as the policy will be forced down the throat.

Turn off jamf
sudo launchctl unload /Library/LaunchDaemons/com.jamfsoftware.jamf.daemon.plist 
sudo launchctl unload /Library/LaunchDaemons/com.jamfsoftware.task.1.plist #com.jamfsoftware.task.{n}.plist check the folder for correct number

Also jamf can do screen sharing, with or without user consent if configured so. It is a RAT as well.


Update
After the daemon being dead for a week, it is back up running again. Mostly the above commands will alone not stop jamf daemon. I doubt it is because of the com.jamfsoftware.startupItem.plist and /Library/LaunchAgents/com.jamfsoftware.jamf.agent.plist. We can check if this malware is running using
ps aux | grep jamf
# root             48968   0.0  0.0  xxx  xxx   ??  SN    0:00   0:00.11 /usr/local/jamf/bin/jamfAgent
# root             48834   0.0  0.1  xxx  xxx   ??  SNs   0:00   0:00.37 /usr/local/jamf/bin/jamf launchDaemon -monitorUsage -enforceRestrictions -monitorNetworkStateChanges
So to get rid of this, rename jamf and not delete because we need it. Or remove the plists.
sudo su
cd /usr/local/bin
mv jamfAgent ripjamfAgent
mv jamf ripjamf
RIP jamf.

Monday, 4 September 2017

[RFC 4226] HOTP for BlackBerry 10 (QNX 6)

Implemented RFC 4226 - HOTP: An HMAC based One-Time-Password algorithm for BlackBerry 10 running QNX 6. Uses BlackBerry Cryptographic Kernel version 5.6 (SB-GSE-56). Various things are pending and the app in the current state can be found at github called QAuthenticator. A reference implementation of the algorithm in Groovy is at hotp.groovy.

I must say, BlackBerry cryptography library is just fantastic. And Qt makes C++ fun to program after me programming in higher level languages like Clojure.

Sunday, 3 September 2017

Linking Libraries in a BlackBerry 10 Cascades Project

Usually when developing C/C++ project we specify linked libraries in a Makefile. With BlackBerry 10 Cascades project, the libraries can be added in the <ProjectName>.pro file. This file is less clouded than the Makefile and easily manageable. For example, if you are developing a crypto app, you need to link the crypto library which is huapi. Add the below to the .pro file.
LIBS += -lhuapi
Else we will get errors like undefined reference to symbol and error adding symbols: DSO missing from command line.
Now we are ready to conquer the world!

Monday, 21 August 2017

Getting Started with Cordova and ClojureScript

This is a small writeup about getting started quickly with Cordova and ClojureScript.

1. Install cordova as usual.
$ sudo npm install -g cordova
2. Create an app.
$ cordova create MyApp
3. Add platform.
$ cd MyApp
$ cordova platform add ios
Cordova app is all set. Now instead of JavaScript we want to use ClojureScript. This part can be treated as a separate project. Write ClojureScript, compile it to JavaScript, place it in the www folder of the cordova project and build the project as usual with appropriate overrides in the respective platform folders.

Install leiningen, Java 8, Clojure 1.8.
1. Create a lein project. (We are inside MyApp folder).
$ lein new my-app
2. Update the project.clj as shown.
(defproject myapp "0.1.0"
  :description "My Lovely App"
  :url "http://example.com"
  :plugins [[lein-cljsbuild "1.1.7"]]
  :dependencies [[org.clojure/clojure "1.8.0"]
                 [org.clojure/clojurescript "1.9.521"]]
  :cljsbuild {
    :builds [{
      :id "core"
      :source-paths ["src"]
      :compiler {
        :output-to "../www/js/myapp.js"
        :optimizations :whitespace
        :pretty-print true}}]})
The source folder is the src in the my-app directory. Now we are ready to write some code.
3. The main file inside src is com/example/myapp/core.cljs
(ns com.example.myapp.core)

(defn foo []
  (println "Hello, World!"))  ; Note we use println rather than console.log

(enable-console-print!)  ; this translates the println to console.log
(set! js/foo foo)  ; export the function declared in this module (namespace) to global
3. Compile the ClojureScript to JavaScript. This will watch for modifications and auto compile every time.
lein cljsbuild auto
4. Include the my-app.js in the index.html file and load it in the browser. Open console and type foo(), which will print Hello, World! to the console.

Now we are in the right path to building lovely apps with ClojureScript and Apache Cordova! But there is a caveat. This hello world app generate 32281 lines of code. I do not see much advantage in using ClojureScript as I am a big believer of being a minimalist when doing front-end development and pretty comfortable with all the craziness that JavaScript offers. Plus this has to run on smartphones and consume less energy but I don't have the luxury of going fully native. So my take, pure JavaScript.

Thursday, 17 August 2017

NPE in getUnmarshallerFactory() in OpenSAML 3

If you are getting NullPointerException when trying to get unmarshaller factory in OpenSAML 3, means most likely OpenSAML 3 has not been initialized. I was calling OpenSAML 3 methods in my unit test suite which gave me the below error. Call (InitializationService/initialize) to initialize the library. My main program does the initialization, so I do not get error when running, but unit test does not invoke that code path.
lein test :only com.concur.saml.saml-test/saml-response

ERROR in (saml-response) (XMLObjectProviderRegistrySupport.java:126)
Uncaught exception, not in assertion.
expected: nil
  actual: java.lang.NullPointerException: null
 at org.opensaml.core.xml.config.XMLObjectProviderRegistrySupport.getUnmarshallerFactory (XMLObjectProviderRegistrySupport.java:126)
    com.concur.saml.core$get_response.invokeStatic (core.clj:253)
    com.concur.saml.core$get_response.invoke (core.clj:249)
    com.concur.saml.saml_test$fn__7679.invokeStatic (saml_test.clj:22)
    com.concur.saml.saml_test/fn (saml_test.clj:19)
 ...
    user$eval85$fn__136.invoke (form-init5596096413821124374.clj:1)
    clojure.lang.AFn.applyToHelper (AFn.java:156)
    clojure.lang.AFn.applyTo (AFn.java:144)
    clojure.core$apply.invokeStatic (core.clj:648)
...
    clojure.lang.AFn.applyToHelper (AFn.java:156)
    clojure.lang.Var.applyTo (Var.java:700)
    clojure.main.main (main.java:37)

Ran 1 tests containing 1 assertions.
0 failures, 1 errors.
Tests failed.

Wednesday, 31 May 2017

Slide - SAML, Variants, Functors, Monads and Exceptions

A presentation I gave at work on SAML, using variants and exception handling in Clojure, functors, applicative functors and monads in Haskell and how Maybe and Either monads short circuits during exception and such. Removed some internal code and links.


The variant C code can be downloaded from Github.
Download this slide from Github.

Friday, 12 May 2017

BlackBerry Passport MicroSDXC Card Support

BlackBerry Passport supports microSD cards upto 128GB. microSDXC cards can also be used with it. However, BlackBerry 10 recognises only FAT formatted external partitions and these cards comes mostly with ExFAT. So the device will show that the media card is not supported and is downloading drivers, but it will fail with an error. To fix this, erase the card and choose FAT as the partition format. Then the OS will recognise the microSDXC card.

Monday, 8 May 2017

Get RSA PublicKey from XML Key Format

Here is a script (prototype) in Groovy to get RSA PublicKey from XML public key. You might encounter such XML keys, say during .NET interop.
import javax.xml.parsers.DocumentBuilder
import javax.xml.parsers.DocumentBuilderFactory
import org.w3c.dom.Document
import java.nio.charset.StandardCharsets
import java.security.spec.RSAPublicKeySpec
import java.security.KeyFactory
import java.security.PublicKey

def rsaPubXML = "ANxn+vSe8nIdRSy0gHkGoJQnUIIJ3WfOV7hsSk9An9LRafuZXYUMB6H5RxtWFm72f7nPKlg2N5kpqk+oEuhPx4IrnXIqnN5vwu4Sbc/w8rjE3XxcGsgXUams3wgiBJ0r1/lLCd6a61xRGtj4+Vae+Ps3mz/TdGUkDf80dVek9b9VAQAB"
def docBuilderFactory = DocumentBuilderFactory.newInstance()
def docBuilder = docBuilderFactory.newDocumentBuilder()

def b64Decode(enc) {
    Base64.getDecoder().decode(enc)
}

Document xmlDoc = docBuilder.parse(new ByteArrayInputStream(rsaPubXML.getBytes(StandardCharsets.UTF_8)))

def modulus = xmlDoc.getElementsByTagName("Modulus").item(0).textContent
def exponent = xmlDoc.getElementsByTagName("Exponent").item(0).textContent
println "modulus: ${modulus}\nexponent: ${exponent}"

RSAPublicKeySpec keySpec = new RSAPublicKeySpec(new BigInteger(b64Decode(modulus)), new BigInteger(b64Decode(exponent)));
KeyFactory keyFactory = KeyFactory.getInstance("RSA");
PublicKey key = keyFactory.generatePublic(keySpec);

println "key: ${key}"
This gives the following output.
modulus: ANxn+vSe8nIdRSy0gHkGoJQnUIIJ3WfOV7hsSk9An9LRafuZXYUMB6H5RxtWFm72f7nPKlg2N5kpqk+oEuhPx4IrnXIqnN5vwu4Sbc/w8rjE3XxcGsgXUams3wgiBJ0r1/lLCd6a61xRGtj4+Vae+Ps3mz/TdGUkDf80dVek9b9V
exponent: AQAB
key: Sun RSA public key, 1024 bits
  modulus: 154774478177095248394968828543369801032226937226535865231262824893513573019304152154974259955740337204606655133945162319470662684517274530901497375379716962851415879364453962123395223899051919634994929603613704222239797911292193776910691509004328773391280872757318122152217457361921195935350223751896771182421
  public exponent: 65537
Note that the modulus must be a positive integer. If you are working with other JVM languages and are getting a negative integer value, specify the signum as 1 in the BigInteger(1, b64Decode(modulus)) function call. The exponent must always be 65537 as of now because that is the largest Fermat's Prime known today.

Sunday, 30 April 2017

Base64 macOS App Release

Released v1.0 of Base64 macOS app. It is a simple app for encoding and decoding base64 texts. It will encode texts as we type or paste. This program does not access any files or network and works offline. It is sandboxed as well.

Encode screen

Decode screen

Source code at GitHub. For downloads, check the release folder.

Saturday, 29 April 2017

Working with AppKit Delegates

Delegates are responders that acts to events that occurs in a program. AppKit delegates often work with Cocoa UI events. Here we will see two examples of handling events, one for NSTextField and another for NSTextView in conjunction with Interface Builder, rather than programatically.

1. Create a macOS Cocoa project from Xcode which will generate an AppDelegate and a ViewController as usual.
2. We will make the ViewController as the delegate to respond to events. For that we need to declare that the ViewController adopts the formal protocol defined by the delegates.
@interface ViewController : NSViewController<NSTextViewDelegate, NSTextFieldDelegate> {
3. Choose the Main.storyboard and choose the View Controller Scene, drag and drop Text View and Text Field components.
4. Choose the Text View from the Document Outline of the View Controller Scene, option click, and in the popup, connect the delegate outlet to the View Controller. Same for Text Field.

5. Now, in the ViewController.h header, declare two IBOutlets which will connect the components in the storyboard to the code.
@interface ViewController : NSViewController<NSTextViewDelegate, NSTextFieldDelegate> {
    IBOutlet NSTextView *textView;
    IBOutlet NSTextField *textField;
}
Since these interface builder outlets are not connected yet, the radio box is in unchecked state.
6. Go back to the interface builder (the storyboard file), choose Text View, option click, drag and connect the New Referencing Outlet to View Controller which brings the above IBOutlets. Choose textView to make the connection. Do the same for Text Field, but here we should choose textField as the referencing outlet.


7. Back to code, open ViewController.m implementation file and implement any of the delegated methods.
#pragma mark - delegates

/* NSTextView */
- (void)textDidChange:(NSNotification *)notification {
    NSLog(@"text did change");
    textView = [notification object];
    NSLog(@"string: %@", [textView string]);
}

/* NSTextField */
- (void)controlTextDidChange:(NSNotification *)obj {
    NSLog(@"control text did changed");
    textField = [obj object];
    NSLog(@"text: %@", [textField stringValue]);
}
The above methods are invoked when the text in a text view or text field changes. The same concept extends to Cocoa Touch and iOS development.

The sample project can be downloaded from github.

Wednesday, 26 April 2017

Call blocking in BlackBerry 10 for a known number

We do not need fancy app to block calls. By default, BB10 has option to block all incoming calls or none. Not individually. But there is a better way. Ideally, we should not be blocking calls, because the caller can identify that a call has been blocked or not. It will ring once and then get busy or some other tones. Better way is to just disable all notifications for a number. For that, first save the annoying number to your contacts and in the "Ringtones and Notifications" option for that contact, choose "Phone Calls" and turn off "All Notifications". That is all there is. Now the call gets received, but you would not know unless you look at the phone. No disturbance.

Monday, 17 April 2017

Slack for BlackBerry 10

There is no official Slack app for BlackBerry 10. Also, if we open a slack channel on the native BlackBerry 10 browser, it just takes to the team settings page. I have figured ways to make it work though, which in fact is quite simple. Quick way is to enable desktop mode in the BB10 browser and then channels load instead of seeing the settings page for the team. But it's not at all user friendly and unsupported browser message will be shown.

Another way is to install the Slack app for Android by sideloading the apk file. This works fine when the team uses email for login. But, when it's configured for SSO only, then we have to follow additional steps. The native browser can open apps that have deep linking registered. Slack does not seem to have this. In order to make this work, we need to use a browser which also runs on the Android runtime. So get the Aurora Browser installed, open the Slack app, choose SSO sign in and this time choose Aurora browser when the dialog pops up. After authentication, a page will be shown with a Download Slack app or Open in Slack app buttons. Choose the later and the Slack app launches, and you get logged in to the team. Enjoy all the awesomeness of the Android Slack app.

I have not checked whether notifications will get received. But I doubt that. So if you don't receive it, you can choose the send email notifications when away for five minutes option from the team settings. Though there is a slight delay in you getting notified, the problem is solved.

I have tried S10 app from BlackBerry World, but for some reasons, I don't get any notifications. Plus the app is rather minimal with text only interface. Kind of have an IRC feel to it.

2FA with SAP Authenticator for BlackBerry 10

The search for a decent 2FA app for BlackBerry 10 is over. SAP Authenticator works perfectly on BlackBerry 10 devices. This is a generic 2FA app and is not tied to any service like Duo Mobile for BlackBerry 10. As in Android, this app requires Barcode Scanner to be installed as well. Instead of Google Authenticator you can use this. It can be used for any 2FA that uses TOTP (Time-based OTP). Happy camper!

Tip: When a website provides options for 2FA app, choose Android as the platform which will display the QR code. Choosing BlackBerry will give the secret key which we have to enter in manually in the SAP Authenticator app for setup rather than scanning a QR code.

Sunday, 16 April 2017

youtube-dl for Muxing Streams

Let's say we need to download videos from youtube with the highest audio and video quality. Sometimes, the combination is not available with most downloaders. But youtube-dl can download separate audio and video streams and mux them together. Here are some commands to do that.
# List all available streams for a video
➜ youtube-dl -F "https://www.youtube.com/watch?v=abcd1234"
Outputs
[youtube] abcd1234: Downloading webpage
[youtube] abcd1234: Downloading video info webpage
[youtube] abcd1234: Extracting video information
[youtube] abcd1234: Downloading js player en_US-vfl5-0t5t
[youtube] abcd1234: Downloading js player en_US-vfl5-0t5t
[info] Available formats for abcd1234:
format code  extension  resolution note
249          webm       audio only DASH audio   56k , opus @ 50k, 1.52MiB
250          webm       audio only DASH audio   84k , opus @ 70k, 2.10MiB
171          webm       audio only DASH audio  126k , vorbis@128k, 3.36MiB
140          m4a        audio only DASH audio  127k , m4a_dash container, mp4a.40.2@128k, 3.69MiB
251          webm       audio only DASH audio  161k , opus @160k, 4.03MiB
278          webm       256x144    144p  101k , webm container, vp9, 13fps, video only, 2.41MiB
160          mp4        256x144    144p  112k , avc1.4d400c, 25fps, video only, 3.19MiB
242          webm       426x240    240p  140k , vp9, 25fps, video only, 2.34MiB
243          webm       640x360    360p  234k , vp9, 25fps, video only, 3.98MiB
133          mp4        426x240    240p  248k , avc1.4d4015, 25fps, video only, 7.13MiB
134          mp4        640x360    360p  254k , avc1.4d401e, 25fps, video only, 6.49MiB
244          webm       854x480    480p  355k , vp9, 25fps, video only, 6.22MiB
135          mp4        854x480    480p  559k , avc1.4d401e, 25fps, video only, 13.74MiB
247          webm       1280x720   720p  601k , vp9, 25fps, video only, 11.70MiB
136          mp4        1280x720   720p 1171k , avc1.4d401f, 25fps, video only, 28.41MiB
248          webm       1920x1080  1080p 1220k , vp9, 25fps, video only, 23.32MiB
137          mp4        1920x1080  1080p 2258k , avc1.640028, 25fps, video only, 57.07MiB
17           3gp        176x144    small , mp4v.20.3, mp4a.40.2@ 24k
36           3gp        320x180    small , mp4v.20.3, mp4a.40.2
43           webm       640x360    medium , vp8.0, vorbis@128k
18           mp4        640x360    medium , avc1.42001E, mp4a.40.2@ 96k
22           mp4        1280x720   hd720 , avc1.64001F, mp4a.40.2@192k (best)
➜
We can see that the highest quality video is 137 with 1080p 2258k and audio is 251 with 161k. But these are video only and audio only streams. Let's combine them.
➜ youtube-dl -f 137+251 "https://www.youtube.com/watch?v=abcd1234"
It will pick a compatible format when merging. If mp4 doesn't work it gets converted into mkv mostly.

Misc
Automatically choose best video and audio: -f bestvideo+bestaudio
Download subtitle if the video has one already uploaded: --write-srt --sub-lang en

Install BlackBerry Blend/Link under macOS Sierra

The installer Install BlackBerry 10 Desktop Software.app inside the BlackBerry 10 Desktop Software_1.2.0.58_B60.dmg image for BlackBerry Link/Blend does not start the installer under macOS Sierra. Running the binary inside the installer .app directly gives the following error:
➜  ~ /Volumes/BlackBerry\ 10\ Desktop\ Software/Install\ BlackBerry\ 10\ Desktop\ Software.app/Contents/MacOS/Install\ BlackBerry\ 10\ Desktop\ Software ; exit;
/Library/LaunchAgents/com.rim.BBLaunchAgent.plist: Could not find specified service
/Library/LaunchDaemons/com.rim.BBDaemon.plist: Could not find specified service
However under the Resources folder of the package content there is BlackBerry Blend.pkg installer. Running it brings up the installer window and the installation succeeds and everything works fine ever after.

Sunday, 2 April 2017

Fun with Hopper on OS X

It has be a while that I touched any assembly code. So I thought I will refresh, and have some fun while at it. Hopper is a disassembler for macOS. It has call flow graphs and pseudo code as in IDA Pro. Another awesome thing is the themes, which adds a modern touch, and is easy to use. Btw., I don't really use IDA Pro. This is more about using Hopper for disassemble and patching Mac OS X binaries. For that I will choose a real app. There was a case where I had to downgrade iTunes once. For that I had to use AppZapper. AppZapper is a paid software. When you open it you can see the nag screen. But removing the nag is simple.



Open the app in Hopper. The AppZapper package has only one binary file AppZapper. Proceed with the default choices, i.e., for the package the loader is FAT archive x86-64 bit and for the executable it is Mach-O 64 bits. It shows the entry point procedure at address 0000000100000f60. Before proceeding, enable "Show the HEX column" to easily see the hex of the instruction set, like in OllyDbg.

Now from the left symbols panel, under Proc. choose -[AZAppController applicationDidFinishLaunching:] the idx of which is 88. Alternatively you can directly go to the address of the proc at 000000010000a250. Choose the pseudo code mode from the toolbar to get an idea of what is happening.



What it does is that it loads the AZRegistrationWindowController and calls the validateExistingRegistrationInformation method whose return value is in rax register. Then it takes the lower bytes of the ax register and performs a bitwise and (the test instruction). If the result is 0, means al is 0, then the zero flag (ZF) is set. If zero flag is not set, then do a local jump to the address at loc_10000a6ac which is at 000000010000a6ac. What we need to do is to take that branch which will then skip loading the AZRegistrationWindowController window. To do that we need to change the jne instruction to je. So the easy way is to click on the jne line and click the hex mode, which will highlight the instruction. The hex for jne/jnz instruction is 85 and hex for je/jz is 84. So double click 85 in instruction 0F 85 FC 02 00 00 and replace it with 84. Go back to asm mode and you can see the updated instruction. Only that it is shown in bytes (db).



Now we need to export the binary. From File menu choose "Produce new executable" and save as AppZapper. Go to applications, control click AppZapper and choose "Show Package Contents" and navigate to MacOS. Now replace the executable with the patched one. If you want to preserve the original then, rename that to something else and copy the patched one as AppZapper. Open the app, and we are no more greeted with the nag screen!

But hey, we still got to register and have only 5 zaps, so we will have to purchase if we need to continue using :). The register option is available from the menu.