Sidebar Menu

Projects

  • Dashboard
  • Research Project
  • Milestones
  • Repository
  • Tasks
  • Time Tracking
  • Designs
  • Forum
  • Users
  • Activities

Login

  • Login
  • Webmail
  • Admin
  • Downloads
  • Research

Twitter

Posts by stumathews
Stuart Mathews
  • Home
  • Blog
  • Code
  • Running
  • Gaming
  • Research
  • About
    • Portfolio
    • Info

Using C# and Fast Connect API

Details
Category: Code
By Stuart Mathews
Stuart Mathews
06.Oct
06 October 2017
Last Updated: 06 October 2017
Hits: 5552
  • Programming
  • Citrix

 

I once needed to use C# to call into the Citrix Fast Connect API. This is how I did it. 

I used the authmanager SDK to enumerate apps.

using System;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.IO;
using System.Net;
using System.Runtime.ConstrainedExecution;

using System.Runtime.InteropServices;
using System.Security.Cryptography;
using System.Security.Permissions;
using System.Security.Policy;
using System.Text;
using System.Xml;
using Microsoft.Win32.SafeHandles;
using Citrix.DeliveryServices.Clients.AuthManager;


namespace UseCtxCredApi
{
  class Program
  {
    static void Main( string[] args )
    {

      // Pass credentials securely to fast connect

      MSV1_0_INTERACTIVE_LOGON logon = new MSV1_0_INTERACTIVE_LOGON();

      logon.LogonDomainName = FastConnectWrapper.InitLsaString("imp");
      logon.MessageType = MSV1_0_LOGON_SUBMIT_TYPE.MsV1_0InteractiveLogon;
      logon.Password = FastConnectWrapper.InitLsaString( "Citrix1" );
      logon.UserName = FastConnectWrapper.InitLsaString( "user1" );

      FastConnectWrapper m = new FastConnectWrapper();
      
      int result = 100;
      LOGONSSOUSER_ERROR_CODE transmittionErrorCode = m.MyLogonSsoUser( ref logon, 1, 1,ref result );

      if(transmittionErrorCode != LOGONSSOUSER_ERROR_CODE.LOGONSSOUSER_OK)
        Console.WriteLine("There was a problem trasmitting the credentials to the single sign-on compoment.");

      // Now enumerate applications

      try
      {
        ConnectionManager.Initialize();

      }
      catch (Exception ex)
      {
        System.Console.WriteLine(ex.Message);
        throw;
      }
      
      int authResult = 0;
      try
      {

        Connection pConnection = ConnectionManager.CreateConnection("HTTP API C++ test");
        LogonStatus loggedOnStatus = pConnection.GetLogonStatus("http://imp-xd1.imp.net/Citrix/Authentication/auth/v1/token");
        if (loggedOnStatus != LogonStatus.LoggedOn)
        {
          Console.WriteLine("Not Logged in");
          return;
        }
        Console.WriteLine( "Logged in" );
       

        string enumeration_result;

        // Now everything is initialised, do some tasks.
        // These may or may not throw, but as good practice expect an AuthManagerException from anything using the SDK.
        
        //enumeration_result = DoResourceAccess();
        enumeration_result = DoResourceAccess_GetSilent();
        if (String.IsNullOrEmpty(enumeration_result))
          return;

        // get first ica launch resource in the enumeration results.
        XmlDocument doc = new XmlDocument();
        doc.Load( new System.IO.StringReader( enumeration_result ) );
        
        XmlNode firstApp = doc.SelectSingleNode("//*[local-name()='launchica'][1]");
        if( firstApp != null)
          PostData( firstApp.InnerText );
      }
      catch (AuthManagerException ex)
      {
        authResult = 2;
      }

      try
      {
        ConnectionManager.Uninitialize();
      }
      catch (AuthManagerException ex)
      {
        authResult = 3;
      }

      if( authResult != 0)
          Console.WriteLine("Problem using the AuthManager SDK.");

      Console.WriteLine("Press any key to perform fast connect Logoff");
      Console.ReadKey();
      m.MyLogoffSsoUser(0);
    }

    // This sample demonstrates making a POST request for an ICA launch
    // allowing authentication prompts,
    // and prints all the HTTP response data to the console if successful.
    static void PostData(string launchuri)
{
    try
    {
        // Create a connection. This object can be reused for the lifetime of this task.
        Connection pConnection = ConnectionManager.CreateConnection("HTTP API C++ test");

        // Create a request. A client must create a new request for each attempt
        // to access resources. (i.e. GetResponse is valid for only one call.)
      using (IAMHttpRequest request = pConnection.CreateAMHttpRequest())
      {
         // Set up the request parameters
        //request.Uri = new Uri( "http://sf3.xd.local/Citrix/SF3/resources/v2/Q29udHJvbGxlci5Ob3RlcGFk/launch/ica" );
        request.Uri = new Uri( launchuri );
        // Set method and upload data
        request.Method = "POST";
        const string requestBody = "<?xml version=\"1.0\" encoding=\"utf-8\"?><q1:launchparams xmlns:q1=\"http://citrix.com/delivery-services/1-0/launchparams\"></q1:launchparams>";
        request.Body = Encoding.UTF8.GetBytes(requestBody);
        // Add any required headers.
        // Note that User-Agent and Connection headers are reserved.
        request.AddHeader("Content-Type","application/vnd.citrix.launchparams+xml");
        request.AddHeader("Accept","*/*");

        // Send the request and get a response
        // Send the request and get a response. This may throw an HttpException or HttpsTrustException
        IAMHttpResponse response = request.GetResponse();
        // Do something with the response data, e.g. Send to std::wcout.
        // Print the HTTP status info.
          System.Console.WriteLine("HTTP "+ response.StatusCode + " " + response.ReasonPhrase);
        
        // Print all headers
        var headers = response.Headers;
        foreach (string header in headers)
        {
          Console.WriteLine(header);
        }

        // Print the response body.
        // This example assumes the response is purely narrow characters
        // (and that sizeof(char) == sizeof(AMByte)), and copies the data
        // straight into to a string.
    
          using (MemoryStream ms = new MemoryStream())
          {
            byte[] buffer = new byte[16*1024];
            
            Stream stream = response.GetResponseBodyStream();
            int r = stream.Read(buffer, 0, buffer.Length);
            while (r > 0)
            {
              ms.Write(buffer, 0, r);
              r = stream.Read(buffer, 0, buffer.Length);
            }
            string icafile = System.Text.Encoding.UTF8.GetString(ms.ToArray());
            System.Console.WriteLine( icafile );
          }
      }

    }
    catch(AuthManagerException ex)
    {
        if(ex.Result == AuthManagerResult.AuthCancelledByUser)
        {
          Console.WriteLine("Cancelled");
        }
        else if(ex.Result == AuthManagerResult.Aborted)
        {
          Console.WriteLine("Aborted");
        }
        else
        {
            // Deal with errors, either from the request or the Auth Manager connection
            Console.WriteLine("Error: " + ex.Message);
        }
    }
}

    
    private static string DoResourceAccess()
    {
      string data = String.Empty;
      try
      {

        // Typically, a client creates and re-uses a single connection object for a task.
        Connection pConnection = ConnectionManager.CreateConnection("HTTP API C++ test");
        LogonStatus a = pConnection.GetLogonStatus( "http://imp-xd1.imp.net/Citrix/Authentication/auth/v1/token" );
        // Create a request. A client must create a new request for each attempt
        // to access resources. (i.e. GetResponse is valid for only one call.)

        using (IAMHttpRequest request = pConnection.CreateAMHttpRequest())
        {
          // Set up the request parameters.
          //request.Uri = new Uri( "http://sf3.xd.local/Citrix/SF3/resources/v2" );
          request.Uri = new Uri( "http://imp-xd1.imp.net/citrix/store/resources/v2" );
          
          // Optionally set the reference URL if the resource URL above
        // is not for a store in Receiver's service records.
          //request.ReferenceUri = new Uri( "http://sf3.xd.local/Citrix/SF3" );
          request.ReferenceUri = new Uri( "http://imp-xd1.imp.net/citrix/store" );
        // Set flags to allow interactive logon if authentication is required.
          request.AuthenticationFlags = AuthenticationFlags.AllowLogon | AuthenticationFlags.Interactive;
        // Optionally set string to append to user agent if needed to identify
        // the Receiver component making the request.
          request.UserAgentSuffix = "AuthManCppSdkSample/5.0";
        // GET is the default method so this next line is not actually required.
        request.Method = "GET";
        // Add any required headers.
        // Note that User-Agent and Connection headers are reserved.
        request.AddHeader("Accept","*/*");
        
        // Send the request and get a response. This may throw an HttpException or HttpsTrustException
        IAMHttpResponse response = request.GetResponse();

        // Do something with the response data, e.g. Send to std::wcout.
        // Print the HTTP status info.
          System.Console.WriteLine("HTTP "+ response.StatusCode + " " + response.ReasonPhrase);
        
        // Print all headers
        var headers = response.Headers;
        foreach (string header in headers)
        {
          Console.WriteLine(header);
        }

        // Print the response body.
        // This example assumes the response is purely narrow characters
        // (and that sizeof(char) == sizeof(AMByte)), and copies the data
        // straight into to a string.
    
          using (MemoryStream ms = new MemoryStream())
          {
            byte[] buffer = new byte[16*1024];
            
            Stream stream = response.GetResponseBodyStream();
            int r = stream.Read(buffer, 0, buffer.Length);
            while (r > 0)
            {
              ms.Write(buffer, 0, r);
              r = stream.Read(buffer, 0, buffer.Length);
            }
            data = System.Text.Encoding.UTF8.GetString(ms.ToArray());
            System.Console.WriteLine(data);
            
          }

        var certificate = response.ServerCertificate;
      }
    }
    catch(AuthenticationException ex)
    {
        // Deal with authentication errors.
        // Alternatively this could be handled by catching just the base
        // AuthManagerException which exposes the same data but may also contain
        // non-authentication error codes. This is shown in the samples below.
        if(ex.Result == AuthManagerResult.AuthCancelledByUser )
        {
          Console.WriteLine("Cancelled");
        }
        else
        {
          Console.WriteLine("Authentication failed: " + ex.Message);
        }
    }
    catch(HttpsTrustException trustEx)
    {
        // There is a trust problem such as an invalid, revoked or expired server certificate.
        Console.WriteLine("A server certificate trust problem occurred: " + trustEx.Message);
        throw;
    }
    catch(HttpException httpEx)
    {
        // There are network problems such as DNS problems or failure to open a connection to the server.
        Console.WriteLine("An HTTP layer problem occurred: " + httpEx.Message);
        throw;
    }
    catch(AuthManagerException ex)
    {
        // Deal with errors, either from the request or the Auth Manager connection
        if(ex.Result == AuthManagerResult.Aborted)
        {
          Console.WriteLine("Aborted");
        }
        else
        {
            Console.WriteLine("Error: " + ex.Message);
        }
    }
      return data;
    }

    // This abbreviated sample demonstrates making a GET request to retrieve a
    // resources enumeration from StoreFront but disallows logon prompts so it's
    // suitable for a non-user initiated background activity.
    // Refer to the sample above for a more complete sample of resource access.
    static string DoResourceAccess_GetSilent()
    {
      string data = String.Empty;
      try
      {
        // Typically, a client creates and re-uses a single connection object for a task.
        Connection pConnection = ConnectionManager.CreateConnection( "HTTP API C++ test" );
        LogonStatus a = pConnection.GetLogonStatus("http://imp-xd1.imp.net/Citrix/Authentication/auth/v1/token");

        using (IAMHttpRequest request = pConnection.CreateAMHttpRequest())
        {
          // Set up the request parameters.
          //request.Uri = new Uri( "http://sf3.xd.local/Citrix/SF3/resources/v2" );
          request.Uri = new Uri( "http://imp-xd1.imp.net/citrix/store/resources/v2" );

          // Optionally set the reference URL if the resource URL above
          // is not for a store in Receiver's service records.
          request.ReferenceUri = new Uri( "http://imp-xd1.imp.net/citrix/store" );
          //request.ReferenceUri = new Uri( "http://sf3.xd.local/Citrix/SF3" );
          // The request is made 'silent' by excluding the
          // AUTHENTICATION_FLAGS_ALLOW_INTERACTIVE flag.
          request.AuthenticationFlags = AuthenticationFlags.AllowLogon;
          // Optionally set string to append to user agent if needed to identify
          // the Receiver component making the request.
          request.UserAgentSuffix = "AuthManCppSdkSample/5.0";
          // GET is the default method so this next line is not actually required.
          request.Method = "GET";
          // Add any required headers.
          // Note that User-Agent and Connection headers are reserved.
          request.AddHeader("Accept", "*/*");

          // Send the request and get a response. This may throw an HttpException or HttpsTrustException
          IAMHttpResponse response = request.GetResponse();

          // Do something with the response data, e.g. Send to std::wcout.
          // Print the HTTP status info.
          System.Console.WriteLine( "HTTP " + response.StatusCode + " " + response.ReasonPhrase );

          // Print all headers
          var headers = response.Headers;
          foreach(string header in headers)
          {
            Console.WriteLine( header );
          }

          // Print the response body.
          // This example assumes the response is purely narrow characters
          // (and that sizeof(char) == sizeof(AMByte)), and copies the data
          // straight into to a string.

          using(MemoryStream ms = new MemoryStream())
          {
            byte[] buffer = new byte[16 * 1024];

            Stream stream = response.GetResponseBodyStream();
            int r = stream.Read( buffer, 0, buffer.Length );
            while(r > 0)
            {
              ms.Write( buffer, 0, r );
              r = stream.Read( buffer, 0, buffer.Length );
            }
            data = System.Text.Encoding.UTF8.GetString( ms.ToArray() );
            System.Console.WriteLine( data );
            return data;
          }
        }
      }
      catch(HttpException httpEx)
      {
        // There are network problems or the server may be untrusted.
        Console.WriteLine("An HTTP layer problem occurred: " + httpEx.Message);
        throw;
      }
        // if interactive authentication was not required then there will be
        // a response available...
      catch(AuthManagerException ex)
      {
        // This error is expected if the silent request couldn't be
        // completed because authentication required a logon prompt to be shown.
        if (ex.Result == AuthManagerResult.AuthInteractionNotAllowed)
        {
           Console.WriteLine("Request couldn't complete silently: " + ex.Message);
        }
        else
        {
          Console.WriteLine("Authentication failed: " + ex.Message);
        }
      }
      return data;
    }
} // Program
  
  class FastConnectWrapper
  {
    [DllImport( "CtxCredApi.dll", EntryPoint = "LogonSsoUser", CallingConvention = CallingConvention.Winapi, CharSet = CharSet.Unicode)]
    private static extern LOGONSSOUSER_ERROR_CODE LogonSsoUser(ref MSV1_0_INTERACTIVE_LOGON pNewCredentials, 
      int bDisconnectCurrentUser, 
      int bRestartPna, 
      ref int pDwResult );

    [DllImport( "CtxCredApi.dll", EntryPoint = "LogoffSsoUser",CallingConvention = CallingConvention.Winapi)]
    private static extern UInt32 LogoffSsoUser( 
      UInt32 timeout );

    public LOGONSSOUSER_ERROR_CODE MyLogonSsoUser( ref MSV1_0_INTERACTIVE_LOGON newCredentials, 
      int disconnectCurrentUser, 
      int restartPna,
      ref int result )
    {
      return LogonSsoUser( ref newCredentials, disconnectCurrentUser, restartPna, ref result );
    }

    public UInt32 MyLogoffSsoUser( UInt32 timeout )
    {
      return LogoffSsoUser( timeout );
    }

    public static LSA_UNICODE_STRING InitLsaString( string s )
    {
      // Unicode strings max. 32KB
      if(s.Length > 0x7ffe)
        throw new ArgumentException( "String too long" );
      LSA_UNICODE_STRING lus = new LSA_UNICODE_STRING();
      lus.Buffer = s;
      lus.Length = (ushort) (s.Length * sizeof( char ));
      lus.MaximumLength = (ushort) (lus.Length + sizeof( char ));
      return lus;
    }
  }
  /// <summary>
  /// Type of SSO logon error code (Citrix)
  /// </summary>
  public enum LOGONSSOUSER_ERROR_CODE
  {
    LOGONSSOUSER_OK = 0,
    LOGONSSOUSER_UNABLE_TO_GET_PIPE_NAME = -1,
    LOGONSSOUSER_UNABLE_TO_CONNECT_TO_SSO = -2,
    LOGONSSOUSER_UNABLE_TO_SEND_REQUEST = -3,
    LOGONSSOUSER_INVALID_RESPONSE = -4
  };

  public enum MSV1_0_LOGON_SUBMIT_TYPE
  {
    MsV1_0InteractiveLogon = 2,
    MsV1_0Lm20Logon,
    MsV1_0NetworkLogon,
    MsV1_0SubAuthLogon,
    MsV1_0WorkstationUnlockLogon = 7,
    MsV1_0S4ULogon = 12,
    MsV1_0VirtualLogon = 82,
    MsV1_0NoElevationLogon = 82
  };

  /// <summary>
  /// Interactive logon structure for use with Citrix API (msdn)
  /// </summary>
  [StructLayout( LayoutKind.Sequential, Pack = 1 )]
  public struct MSV1_0_INTERACTIVE_LOGON
  {
    public MSV1_0_LOGON_SUBMIT_TYPE MessageType;
    public LSA_UNICODE_STRING LogonDomainName;
    public LSA_UNICODE_STRING UserName;
    public LSA_UNICODE_STRING Password;
  }

  [StructLayout( LayoutKind.Sequential, CharSet = CharSet.Unicode )]
  public struct LSA_UNICODE_STRING
  {
     public ushort Length;
     public ushort MaximumLength;
    [MarshalAs( UnmanagedType.LPWStr )]
     public string Buffer;
  }

  internal class FastConnectLibraryWrapper
  {
    [DllImport("FastConnectLib.dll")]
    internal static extern UInt32 LogoffSsoUser( UInt32 timeout );

    [DllImport( "FastConnectLib.dll" )]
    internal static extern LOGONSSOUSER_ERROR_CODE LogonSsoUser( ref MSV1_0_INTERACTIVE_LOGON pNewCredentials,
                                              int bDisconnectCurrentUser,
                                              int bRestartPna,
                                              ref UInt32 pDwResult );

    public UInt32 callLogoffSsoUser( UInt32 timeout )
    {
      return LogoffSsoUser( timeout );
    }

    public LOGONSSOUSER_ERROR_CODE callLogonSsoUser( ref MSV1_0_INTERACTIVE_LOGON pNewCredentials,
                                              int bDisconnectCurrentUser,
                                              int bRestartPna,
                                              ref UInt32 pDwResult )
    {
      return LogonSsoUser( ref pNewCredentials, bDisconnectCurrentUser, bRestartPna, ref pDwResult );
    }

  }

}

Thoughts on Perl, Git and Windows 10

Details
Category: Code
By Stuart Mathews
Stuart Mathews
19.Aug
19 August 2017
Last Updated: 05 October 2017
Hits: 2417
  • Perl

Been reading up on Perl, I’ve already made use of a bunch of functionality in it but I’d like to go to the next level. Perl to me is like the fast equivalent version of C – fast in terms of productivity and time to write software not execution speed – C is kind in this regard. I read a bit today on the the Win32 extensions but I don't think I’m that interested right now in it. I think what I’d like to get a better grip on now is the data reporting side of things.I’d like to use my knowledge of regular expressions in the vehicle that made regular expressions famous – Perl. I just need a project and the inclination. Everything else is vying for my attention but I’ll come around…

Citrix has started using git now. We used to use perforce but we’re slowly moving away from it. I like perforce and well I also liked subversion. Safe to say git is pretty ok too. The guys are slowly moving to git and around the office I’m hearing the usual, “but I used to do this in perforce so easily!”. I’ve been using git well before my work required me to so that's a good thing so the switch is easy.

There are a few things that I’ve noticed that I took for granted with perforce – visual diffs, time lapsed views, searching submitted change lists etc. Obviously the GUI of P4V made things a whole lot easier. And I’m finding the visual studio git integration pretty good also. I use it mainly for visual diffs and for staging and committing. I sometimes use Source Tree but not much.

I’m sure I’ll be hearing more moans and groans in the next couple of weeks until they realise how awesome it is.

I still find windows 10 sucky. I’ve always thought the windows start menu and metro interface is a step backwards. I’m updated to the Creators update and well nothing much has changed. Microsoft really shot themselves in the leg with Windows 8,8.1 and 10. Totally misinterpreted the use of a PC in a mobile perspective – trying to merge cell phones and PCs and the experience sounds cool but the implementation is fudy-dudy. They’ve really made Linux look good though, I’m pleased about that. Ubuntu and fedora are now not as ugly as the rest of the world, its just that the rest of the world got uglier!

Doing things in Perl

Details
Category: Code
By Stuart Mathews
Stuart Mathews
16.Jul
16 July 2017
Last Updated: 05 October 2017
Hits: 2777
  • Programming
  • Perl

I've been learning perl recently. Particualrly to re-write a program i wrote in both c# and java. The program is supposed to download stock prices form the internet in real time.

The biggest problem out there is that there aren't too many(actually I could only find one) free services that allow you to query stock information. Yahoo finance is the best one because its quite usable. There is a google service which they've discontinued. I guess, you'd get everything you wanted if you paid for it.

Anyway, the issue, once finding a service to query(yahoo) was figuring out which language to use to write it in. I naturally turned to C# because that's what i use day-to-day in my job. The issue with this was that this was actually written for my dad, who is quite forward thinking and doesn't use Windows. Sure, I guess one could use Mono on Linux but I'd not tried that recently. He also has a Mac(unix) so I figured Java would be a cross platform alternative. So I wrote it in Java. I also secretly wanted to write it in java. 

The issue I had was that writing these solutions in a compiled language like c# or java means, well its a black box - you can't really extend it yourself without re-compiling it - something I'd do but perhaps no one else would like to do. So adding new features mean re-coding and re-compiling. Not the end of the world. I even thought about writing the software in C or C++ but though better of it(Although I love C). I thought better of it because of the amount of work which would be involved: you don't get advanced data structures in C by default, I guess you do with c++ but then you'd be compiling it and sealing up the solution again and it would be difficult to modify once its been deployed.

Actually, none of this really, hugely matters to me but because they are some reasons...and i only need one - i decided to do it in a scripting language. Perl is a scripting language with great support for lots of 'extras' that make it feel like a compiled language, which is usually has a host of libraries that make the work easier - things like parsing JSON, making HTTP requests etc.. Perl has this, and there was a big book on Perl in passing on a library book shelf, so that helped too.

I'll post the C# and Java versions just for fun later on but here is the Perl script (its highlighted in Ruby syntax - i dont have a perl highlighter at the moment)

#!/usr/bin/perl
use strict;
use warnings;
use LWP::UserAgent;
use Storable;
use JSON qw( decode_json );
use Scalar::Util qw(reftype);
use Parallel::Iterator qw( iterate_as_hash );
use Data::Dumper;
use URI::Encode qw(uri_encode uri_decode);
use Getopt::Std;

my %options=();
# -t 4, -d "delimiter" -o "outputfile.csv" -r "us" -l "en-gb" -v -x "exclude.csv" -l 5(co limit)
getopts("ht:d:o:r:l:vx:bl:", \%options);
my \(verbose = \)options{v};
my \(colimit = \)options{l};
if(\(verbose) {
	foreach my \)opt(keys %options) {
		print "\(opt = \)options{\(opt}\n";
	}
}
if(\)options{h}){
	print "./endofday.pl -t <numThreads> -d <delimiter> -o <ouputfile> -r <region> -l <language> -x <exclude file> -l <limit> -v\n";
	exit(0);
}

sub ConvertCompanyToTicker {
	my @args = @_;
	my \(company = uri_encode(shift @args);
	my \)region = shift @args || (\(options{r} || "us");
	my \)lang = shift @args ||  (\(options{l} || "en-gb");
	my \)json = getJson("http://d.yimg.com/aq/autoc?query=\(company&region=\)region&lang=\(lang");
	if (\)json) {
	    my \(jObj = decode_json(\)json);
	    my @queryResult = @{\(jObj->{'ResultSet'}{'Result'}};
	    for my \)var (@queryResult) { 
		    return \(var->{symbol};
	    }
	}
	return undef;
}

sub ConvertTickerToStock {
	my @args = @_;
	my \)ticker = shift @args;
	print "Live ConvertTickerToStockTicker: '\(ticker'\n";
	my \)url = "https://query.yahooapis.com/v1/public/yql?q=".
		           "select%20*%20from%20yahoo.finance.quotes%20where%20symbol%20in%20(%22" .\(ticker.
			   "%22)&format=json&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys&callback=";
	my \)json = getJson(\(url);
	if(\)json) {
		my \(jObj = decode_json(\)json);
		my \(queryResult = \)jObj->{'query'}{'results'}{'quote'};
		return \(queryResult || undef;
	}
	return undef;
}

sub getJson {
	my @args = @_;
	my \)url = shift @args;
	my \(req = HTTP::Request->new(GET => \)url);
           \(req->header('content-type' => 'application/json');
	my \)ua = LWP::UserAgent->new;
        my \(resp = \)ua->request(\(req);

        if (\)resp->is_success) {
	    return \(resp->decoded_content;
        } else {
            print "HTTP GET error code: ", \)resp->code, "\n";
            print "HTTP GET error message: ", \(resp->message, "\n";
	    return undef;
        }
}

#Global store of all tickers and their stock details
my %all;

#Read in Cache of ticker to company names to prevent relookup(expensive)

my \)cacheFileName = "co2tick.cache";
my \(progressCacheFileName = "progress.cache";
my \)haveTickerCache = -e \(cacheFileName;
my \)haveProgressCache = -e \(progressCacheFileName;
my %resolutionCache;
if(\)haveTickerCache) {
	%resolutionCache = %{ retrieve(\(cacheFileName) };
}
if(\)haveProgressCache) {
	%all = %{ retrieve(\(progressCacheFileName) };
}

\)SIG{'INT'} = sub {
	store(\%all,\(progressCacheFileName);
	exit 1;
};

# read in all the companies and exclude the ones in the exclude file
my @companies = <>;
my \)excludeFile = \(options{x};

if(\)excludeFile && -e \(excludeFile) {
	print "using exclude file '\)excludeFile'\n" if(\(verbose);
	open (EXCLUDE, "< \)excludeFile") or die "Can't open \(excludeFile for read: \)!";
	my @lines = <EXCLUDE>;
	my %exclude;
	\(exclude{\)_} = undef foreach (@lines);
	# exclude from companies those that are in exclude file
	@companies = grep {not exists \(exclude{\)_}} @companies;
	close EXCLUDE or die "Cannot close \(excludeFile: \)!"; 
}

# process companies
my \(lineCount = 0;
foreach my \)line(@companies) {
	my \(company = \)line;
	chomp \(company;
	chop \)company;
	next if !\(company;
	my \)ticker;

	# We're going to store company names with space to underscores so we can have one-worded companies in the cache
	\(company =~ s/ /_/g;
	\)ticker = \(resolutionCache{\)company};

	# get a ticker	
	if(!\(ticker) { 
		\)company =~ s/_/ /g;
	    	print "LIVE convertCompanyToTicker Company:'\(company': ";
		\)ticker = ConvertCompanyToTicker(\(company);
		print (\)ticker || "could not resolved to a ticker symbol.");
		my \(addbadTickersToExcludeFile = \)options{b};
		if(\(addbadTickersToExcludeFile && \)excludeFile && !\(ticker) {
			#exclude bad tickers
			open(my \)ex, '>>', \(excludeFile) or die "Could not open file '\)excludeFile' \(!";
			print \)ex "\(company\n";
			close \)ex;
		}
		print "\n";
		next if(!\(ticker);
	};

	\)all{\(ticker} = undef if(!\)all{\(ticker});

	# Put the multiword comany name in the hash that tracks all the results
	\)company =~ s/ /_/g;
	\(resolutionCache{\)company} = \(ticker;

	last if (\)colimit && (\(lineCount++ == \)colimit));
}

#persist the cache of company to ticker hashes for future lookups...
store(\%resolutionCache,\(cacheFileName);

my \)numThreads = \(options{t} || 2; 
my @columns;
my %output = iterate_as_hash({ workers => \)numThreads },\&ConvertTickerToStock, \%all);
%all = (%all, %output);

# Write all to CSV as output...
open(my \(csv, '>', \)options{o} || 'stocks.csv');
foreach my \(ticker (sort keys %all) {
	my \)delim = \(options{d} || ";";
	my \)stock = \(all{\)ticker}; 
	
	#Get the first stocks values' order as default column order for all following stocks for csv format
	if(!@columns) { 
		@columns = sort keys(%\(stock);
		my @preferred = qw(Name Currency Ask Open PreviousClose PercentChange PriceBook Change DaysHigh DaysLow EarningsShare);
		my @newColumnsWithout = grep {!/join("|",@preferred)/} @columns;
		my @reOrder = (@preferred, @newColumnsWithout);
		@columns = @reOrder;
		print \)csv join(\(delim, @reOrder)."\n";
	}
	my @line;
        
	foreach my \)key(@columns) {
		my \(column = \)key;
		my \(data = \)stock->{\(key} || "none";
		push(@line, \)data);
	}
	print \(csv join(\)delim ,@line)."\n";
	@line = undef;
}
close(\(csv);	
unlink \)progressCacheFileName;

my \(end_run = time();
my \)start_run = \(^T;
my \)run_time = \(end_run - \)start_run;
print "Job took $run_time seconds\n";


This isn't as feature rich yet as the c#/java alternatives because its missing some features but the core is there and I'll add those features when I'm good and ready! 

I've also added new features such as a progress cache and the ability to ctrl-c and abort and then continue later on. Also some other caching is in place to speed up the execution of the program. In fact, I'm dealing with an issue now where its TOO FAST(can you believe it!). I've querying the yahoo API using parallel threads*(http://search.cpan.org/~andya/Parallel-Iterator-1.00/lib/Parallel/Iterator.pm)  and the yahoo service doesn't like that and starts issuing a HTTP 999 which probably means its thinks I'm doing a Denial of service on it! So I'll be doing some more work to make this better - this extensible and easily modifiable way to write code makes solutions progress very easily.

What I do find rather impressive about perl is that this is substantially shorter than the C# and java versions and a lot simpler. 

I like Perl and I'd like to lean more. I have a university course on data visualisation which uses Python and even though i new this, Perl has an alluring thing about it - maybe its because its got such a long legacy in the Unix world and here is the real reason...I really like it's support for regular expressions and I've been yearning for a fast prototyping language that is quick, powerful and when it comes to text, is 2nd to none.

PS: I should probably add more comments but I love seeing the code so sparse! I'll probbaly comment more about this and perl syntax I've used in general moving forward. 

More Articles …

  1. Driven
  2. The C# COM C/C++ divide
  3. Cordova
  4. Virtual channel success
  5. Virtual channel
  6. Simplicity
  7. Named pipes, gaming and COM
  8. Glib and SQL
  9. Broker
  10. Broker architectural design pattern implementation

Subcategories

Game Development Article Count:  28

I discovered the realms of game development purely by accident, having picked up a book entitled 'Core Techniques and Algorithms in Game Programming' and discovered a surprising niche of innovation in programming quite unparalleled to my day-to-day needs as a developer. Here optimisation, graphics rendering, and algorithms are used on a totally different level and its very interesting.

  • 7
  • 8
  • 9
  • 10
  • 11
  • 12
  • 13
  • 14
  • 15
  • 16

Page 12 of 17

Blog RSS Feed