Today I've had to learn what exchange rates are.

I've never really cared about them, not even when exchanging money before holiday and even then not really considered it much more than something to buy at a cost(which I'd not quibble at) - I'd just buy and go. 

I know roughly what they are but I've never really considered them. 

Today I considered them.

Specifically, what I wanted to know was what they actually mean not just that they are "how much of this gets me some of that", which is essentially what it means.

Currency Exchange rates are rates at which a currency exchange will convert from one currency to another.

For example converting from ZA currency to USD currency.

* The ZA to USD exchange rate is expressed as how many USD will it take to convert to 1 ZA (\(0.08)
* The USD to ZA exchange rate is expressed as how many ZA will it take to convert to 1 USD (R12.6)

Note: The the TO Currency is
- What the resulting rate is in. (ZA-US: \)rate) and (US-ZA: Rrate)
- How many TO currency units make up 1 FROM unit (GBP/US - how many USD make up a GBP)

* So the "Exchange rate" for ZA to USD (aka AZ-USD) is how many USD will it take to get to 1 ZA.
* So the exchange rate is very much centered around USD as its all about how many USDs will it take to get to 1 of the Target currency (ZA)

Thus as it will take 0.08 USD to get 1 ZA so that ZA-USD exchange rate is 0.08.

Exchange rates thus are usually reported and stored in the form of FROM_CURRENCY to US:

* EUR/US = \(1.18
* GBP/US = \)1.34
* ZA/US = $0.08
* US/EUR = E0.848
* US/GBP = £0.74
* US/ZA = R12.57

Cross Rates

So its now clear to me that exchange rates quotes are rates that are mostly in respects to USD such as GBP-USD meaning how many USD does it take to get 1 GBP which is: 1.34 (this is also the rate)

If you have two exchange rates stored in this way - rates respective to USD ie

- GBP/US (USDs to make up 1 GBP)
- EUR/US (USDs to make up 1 EUR)
- ZA/US (USDs to make up 1 ZA)

If you want to know the GBP/ZA ie the amount of ZAs that equates to 1 GBP and the above 3 exchange rates are available to you(IE that are all rates showing how many USDs it takes to get to the Target currency), you can calaculate two currencies this way:
* (GBP/US rate) / (ZA/US rate) = 1.34 / 12.57 = R rate = R16.84

If you want to get the exchange rate from one currency to another where both rates are not based on USD, you need to perform another step.

Summary 

- Exchange rates are usually stored and quoted to show how many USD it takes to equate to the currency in question such ie EUR as EUR/US, YEN as YEN/US and GBP as GBP/US

I’ve been working a lot in python recently.

Besides at work, I’ve also been working on converting my broker project which was originally written in C to python.

While doing this, I started to look at the network socket code first and I'm amazed how easily its able to serialize data over the wire using JSON with so little fuss. I'll explore this a bit further...

Now originally I didn't use JSON in the C version, I used MsgPack which is like JSON but it's a binary protocol and faster. Anyway, it roughly achieves the same goal - sending messages across the wire...So

One thing you always need to do in TCP programming is agree on a protocol between the two parties in the communication. Something like "I'll send the size of the packet first, and then the rest of that data...", then the receiving end should read only that much data.Also, you need to convert the bytes to network-byte order before sending and then converting to host order on receiving.

Anyway, you can't really get away from this whether you're in C# or in plain old C or Python for that matter. So just for interest sake, here is how its done in C compared to how its done in Python.

C:

/* readn - read exactly n bytes */
int netReadn( SOCKET fd, char *bp, size_t len)
{
	int cnt;
	int rc;

	cnt = len;
	while ( cnt > 0 )
	{
		rc = recv( fd, bp, cnt, 0 );
		if ( rc < 0 )				/* read error? */
		{
			if ( errno == EINTR )	/* interrupted? */
				continue;			/* restart the read */
			return -1;				/* return error */
		}
		if ( rc == 0 )				/* EOF? */
			return len - cnt;		/* return short count */
		bp += rc;
		cnt -= rc;
	}
	return len;
}

Python:

  length_str = b''
  char = socket.recv(1)
  while char != b'\n':
    length_str += char
    char = socket.recv(1)
  total = int(length_str)
 
  view = memoryview(bytearray(total))
  next_offset = 0
  while total - next_offset > 0:
    recv_size = socket.recv_into(view[next_offset:], total - next_offset)
    next_offset += recv_size 

The python code comes from the jsonsocket source while the previous comes from code in Stulibc.

That was a little diversion that I found quite interesting while converting my C to Python.

Apart from that short little diversion, I’ve also been working generally in python recently at work and made some interesting code...

Chunking-up data before sending it up to the internet. This is very cool, basically turns one array into many smaller chunk-sized arrays. I like.

        if(shouldChunk):
            chunks = numpy.array_split(array(holdings), ChunkSize)
        else:
            chunks = numpy.array_split(array(holdings), 1)

Using list comprehension in python to construct objects in one-line, much like LINQs .Select() function:

requests = [models.HoldingDto(security_uid=request["securityUid"], holding_type=request["holdingType"], units=request["units"], settled_units=request["units"], cost=0, properties=None, transaction=None ) for request in chunk ]      

Caching and reloading of data. In python its a doddle to dump and reload dictionaries and lists. It always amazes me how much we can do with simple lists. Here I'm serializing a list of dictionaries called ticker_to_isin having generated that list using a routine. Which, I need not be redone if we can cache the results...which we can:

    if(isNewLoad or shouldResolveLusid):
        isin_to_secuid = GetSecuids(ticker_to_isin, client)
        pickle.dump(isin_to_secuid, open(secuid_cache_name,"wb"))
    else:
        isin_to_secuid = pickle.load(open(secuid_cache_name,"rb"))
    return (ticker_to_isin, isin_to_secuid)

I've also had to call into Thompson-Reuters data scope platform recently. Basically to get information about RICs - Reuters Instrument Codes which I've come to learn are how Thompson Reuters identifies securities.

Here is how you can obtain a token and issues REST requests to data scope. The link above is to their documentation page which I used to come up with the code below. So, below I'm requesting Isin codes for Ticker aka RIC codes.

This also shows how easy it is to manipulate headers and send plain HTTP requests in python much like in Typescript of Javascript. 

def GetDataScopeToken(username = "x", password = "y"):
    headers = {'Prefer': 'respond-async', 'Content-Type': 'application/json'}
    url = 'https://hosted.datascopeapi.reuters.com/RestApi/v1/Authentication/RequestToken'
    json = { 
        "Credentials": { 
            "Username":  username, 
            "Password":  password
         }
        }
    r = requests.post(url, json = json, headers=headers)
    return r.json()    

def GetDataScopeInstrument(token, source):    
    headers = {'Prefer': 'respond-async', 'Content-Type': 'application/json', 'Authorization': 'Token {token}'.format(token=token)}
    url = 'https://hosted.datascopeapi.reuters.com/RestApi/v1/Extractions/InstrumentSearch'
    json = {"IdentifierType":"Ric","Identifier":source,"InstrumentTypeGroups":["CollatetizedMortgageObligations","Commodities","Equities","FuturesAndOptions","GovCorp","MortgageBackedSecurities","Money","Municipals","Funds"],"PreferredIdentifierType":"Isin","MaxSearchResult":10}
    r = requests.post(url, json = json, headers=headers)
    return  r.json()

I've also discovered an interesting library (tqdm) which allows you to track iterative processes in a visual manner. Whats more is that it works with pythons concurrent.futures library to aid async code which makes me weak at the knees!. I like!

It's just a matter of tacking the functionality in for loops and boom - progress bars! Incredible.

Notice that I'm sending up 10,861 items and I'm dividing this into 200 item chunks, which is roughly 54 items a request. parallelize these and you're in business! 

Here is how you can do it using a tqdm iterator:

iter = tqdm(HoldingDataDf.groupby("HoldingsDate"))
        for group_name, group in iter:
            iter.set_description("Processing Group ({name}), holdings = {size}".format(name=group_name, size=(len(group))))
            # do work

 And here is how you can do it using concurrent.futures:

with concurrent.futures.ThreadPoolExecutor(max_workers=MaxThreads) as executor:
                futures = [executor.submit(SendHoldingsThreadFunc, chunk) for chunk in chunks]   
        
                kwargs = { 'total': len(futures), 'desc' : 'Uploading data' }

                for f in tqdm(as_completed(futures), **kwargs):
                    f.result();

Basically, you pass the futures to the tqdm() function when they are completed.

One-line assignments. I like these. These are so useful because they are concise. I use them when I'm reading in command line switches:

InputFile = options['-i'] if '-i' in options else "HoldingsSummary.xml"
    MaxThreads = int(options['-t']) if '-t' in options else 2
    ShouldResolveTickers = True if '-k' in options else False
    Verbose = True if '-v' in options else False
    Scope = options['-s'] if '-s' in options else "tr"
    ChunkSize = int(options['-u']) if '-u' in options else 200
    DoSynchronous = True if '-0' in options else False
    show_holding_count_by_date = True if '-c' in options else False
    isNewLoad = True if '-n' in options else False  
    shouldResolveLusid = True if '-l' in options else False
    shouldChunk = True if '-p' in options else False
    start = int(options['-x']) if '-x' in options else None
    end = int(options['-y']) if '-y' in options else None
    dryRun = True if '-a' in options else False
    timeout = int(options['-r']) if '-r' in options else 100

Cool huh?

Pythons humble format() makes life so much more manageable. Thanks!

print("python {program_name} -i <inputfile>".format(program_name=program_name))

Here is something I've not spent too much time on but its interesting none-the-less: Pythons ability to define types of arguments and return values:

def TickerToIsin(ticker:str) -> str or None:
    result = GetDataScopeInstrument(token['value'], ticker)
    value = result['value']
    if( len(value) > 0):
        isin = value[0]['Identifier'] 
        print('{ticker}={isin}'.format(ticker=ticker, isin=isin))
        return isin   
    else:
        return None

See how it's saying, ticker is a string and that TickerToIsin() returns a string or None. Usually, you don't do this in python you just return whatever as whatever - just like in Javascript. But you can add this "typedness" if you like it seems. 

Thats it for now :-)

 

 

I finished reading The Lean Startup by Eric Ries on the train this morning. I recently started reading it on the train to and from my new work recently. I've learned some important things which which stood out for me, those aside the underlying message is roughly this:

If you want to survive as a startup, in the uncertain world that it finds itself in, you need to basically a) drop the ego, b) track the right metrics that indicate progress and c) constantly experiment/innovate to indicate the success or failure of your time and efforts. I think its the last one which is the most interesting to me because I think its the most important and the most difficult to quantify and qualify. I left out one: Keep trying to find a way - whatever it takes.

These ideas dovetail with other ideas like, in experimenting often, you increase the feedback loop, you learn quicker and fail quicker etc.

This is all not new to modern software developers. We do this all the time. The Agile process we use in software development, is probably also based on the LEAN methodologies that came out of Toyota etc - particularly the concept of developing MVPs(Minimum viable products) and short cycles of iterative development.

Here are 50 of my favourite and important ideas that stood out for me:

1. You need to measure progress in extreme uncertainty.
2. Measuring productivity will be diffirent in a start-up.
3. There should be an emphasis on short iterations, deliverable products - the AGILE/LEAN Methodology
4. Tinkering at something is a great way to fine tune and find niggles. Do experiments using your experimentation system
5. Many startups are more likly to plan to launch a rocket ship than to plan to drive a car. (Over-complicated/over-engineering)
6. Details are important but over-arching assumption proving/debunking is better. 
7. The Build-measure-learn process helps tuning the engine.
8. Experiment all the time - this is a valuable, provided its with the customer in mind.
9. It is important to build an innovation factory which is based on an experimentation system.*
10. We must learn what customers really want, not what they say they want or what we think they should want.
11. "Validated learning is demonstrating that a team has discovered valuable truths about a startup's present and future business prospects."
12. A telephone is not useful by itself but becomes useful when everyone has one. (having the only telephone in the world isn't useful)
13. Ones predictions and expections can be wrong. Save space and make provision for wrong assumptions and dont tie down your strategy such that you cannot change it easily in the face of these changes.
14. Make your strategies flexible.
15. Customers can refuse to use/realise your great view of the world and can quite happily reject it at a whim - dont assume they will see it your way.
16. Learn important insights from your customers.
17. Aim for value-creating efforts, align them with your customer's problems.
18. Value is defined as providing benefit to the customers
19. Incorrect strategic assumptions happen - expect that you've made some unchecked assumptions.
20. Lay out all your assumptions on the table, routinely*
21. Discover flaws in our assumptions by testing them with experiementation
22. Experimentation is innovative.*
23. Learning must be backed by customer data and validatable*
24. What customers would accept is diffirent to what you think they might.
25. Assumptions are the root to all evil.
26. Ready-to-fail experiments are important.
27. Figuring what to build is a startup's dillema
28. Perform experiments to answer questions and assumptions, preferably one for assumptions/answer required.
29. Experiments take a lot less time than stratagic planning does, its usually not a waste of time.
30. Success is not delivering a feature, its solving a customers problem
31. Experiments are the basis for learning with the customer in mind - a mechanism to validate your time.
32. Learning milestones need to be sufficiently shared and exposed - Latest progress with Excel JavaScript plugin and Reflections on Excel JavaScript Add-in but sadly not exposed sufficiently.
33. "Innovation accounting" in a nutshell is 3 steps MVP+Tune=Pivot Or Abandon.
34. Basing choices on first hand understanding of customers is important
35. Way back, the average consumer at that time was not conversant with personal computers to know if they want to use them in a new way ( or even use tem at all)
36. Any additional work beyond what is required to start learning is a waste.
37. Messy results and bad quality test results are better than assumptions - tests can be messy and that's ok if they validate assumptions in either direction.
38. The only way to win is to learn faster than anyone else.
39. Have a Dose of reality
40. Are you making the product better? Yes? How do you know?
41. Disruptive innovation is what you are aiming for*
42. Test your riskiest assumptions first.
43. Validated learning kanban board is interesting - turn 'done' into 'validated'
44. Every story in validated kanband must be validated - stories need to provide answers to the question - how can this be validated.
45. All Validation needs to be with the customer in mind
46. Best practise can be a waste of time - we're a startup not an industry vetran
47. Working in small batches is better (Finish a complete envolope before moving to the next one) - easier to change strategy
48. Reduce your work in progress - its clusttering your time, mind and its wastage if unresolved.
49. An investment should be small in something that brings small pain, while larger in those that bring long or larger pain. (L EAN - just in time) : proportional investment approach.
50. Critisism is not damaging, its important as a way to adapt or continue. Don't be overly defensive.

And then came a revalation:

A common practise is for the inventor of a new product or feature to manage the subsequent resources, team or division that ultimately commercializes it. As a result, strong creative managers wind up getting stuck working on the growth and the optimization of products rather than creating new ones. This is why established companies struggle to foster innovation in the first place.

 

• Concious effort is important
  Insist that assumptions be started explicitly and tested rigorously out of a genuine desire to discover the truth that underlies a project's vision.

And now for my master drawings... which tries to bring most of all this together for me:

Which is more comprehensibly squashed into these 4 points:

• Uncertainty as input into experiments.
• Experiments drive innovation and test assumptions
• Experiments with the customer in mind and assessing assumptions is a way of calibrating R&D
• Aim to do just what is required, to reduce waste on that which you did but was not required(waste)

 A little word about innovation

Research and development then can be seen as value obtained from researching and developing stuff, and its valuable. To quantify its value you need to define your two points in your measurement strategy, for me having read this book, those would be all about how the research and development benefits the customer.

Its easy and in my opinion wrong, to say its all about anything that helps us(the company, the development team, the sales person) achieve our goal or our vision. This gets you nicely trapped in thinking that everything you do is worthwhile because everything you do you deem important enough to consider ad do in the first place - you self validate your ideas and actions. But its puts out of sight the customer and focuses on what you've already assumed: That this will obviously benefit the customer.

Assumptions are bad. Bring every choice you make relative to the customer. Every piece of research and development needs to satisfy the questions. Does this benefit the customer, In which way will the customer smile at this? Will they notice it? Is it that important fo the customer that we do this. Customer customer customer. Now obviously there will be folks that don't exclusively see this point as useful. That's how you can measure research and development: its how useful that work is to the customer. If its not useful, then your wasting your research and development budget.

I think that no matter the task that any employee is doing, whether that be in software development, sales or accounting - a simple universal question, in plain English, "how will that/this benefit the customer?" could be asked of anyone to anyone irrespective of their expertise in whatever areas they are in. If a software engineer can be asked that about what he is working on by an accountant and they both agree - this is then alignment with customer's needs. That's why everything should be realitive to a customer's needs.

The next thing I'd like to talk about is experimentation and innovation. To innovate in many ways includes finding new ways to succeed. Anyone who can answer the question, what are do doing to innovate most will be unsure how to answer the question - unless they've been working on something new - an innovative project. They can then easily answer that question because its in the forefront of their minds. The time and effort they put into their own projects they belive in constantly answer the questions "why is this useful, why is this useful" over and over again while they are doing it and as they discover new things and enjoy it, the constantly answer that question "this is why, this is so cool". This is innovation. Now aligning this with creative endevour with a companies objectives (one of which is a customer's problems being resolved) is how innovation is done. Innovation projects are like 'inventions' and each person participating is an inventor and thats how they feel bound and commited to that idea of whatever they are doing - thats how innovation endues.

Towards the end of the book and once I felt that i had obtained the needed messages that the book tries to convey, I felt this it did drag on a bit.

I think the most important part of the book for me is fairly evident - its where I stopped making notes and scribbling in my book. For me this was around Part I(Vision), though I did some some scribbles in and around part II(Steer) but found it and part III(Accelerate) quite laborious. This is personal of course, those topics didn't particularly resonate with me, though possible quite important and relevant to others.