DETERMINING TIME DELTAS IN DOS SHELL

I ran into a bit of a tricky problem the other day and I wanted to share it with you.  I am using a program called ATRT (Automated Test and Re-Test) to test our software remotely, and we ran into a problem where two OCR (optical character recognition) extracted time stamp needs to be compared to find a time delta.  Unfortunately, in this version of software we are unable to convert the text time stamp back into a time object, and the software does not currently have a full math library (ex. Modulus function) or token splitting capabilities to determine the delta in-house.  Since it will be a at least a few weeks before we could get these features we decided to find a work-a-round; and, since ATRT can spawn child processes with custom arguments and read in the results we figured a simple Dos Shell Script was the way to go (note - powershell, unix tools, etc... will not be available on the target systems).

Simple idea, of course it's been a few years since I've been forced to use the Windows Command Prompt and the idiosyncracies can definitely lead to some frustrations.  It took about a minute to double check some syntax and write up the script and then to do the testing... first let's look at some code:

   REM > time_delta.bat <start> <stop>
   REM > time_delta.bat 2:03:04 2:05:06
   REM 122

   set MY_START=%1%
   set MY_STOP=%2%

   REM grab the hours, minutes and seconds from the start time fields 1 through 3
   REM grab the hours, minutes and seconds from the stop time  fields 4 through 6
   FOR /f "tokens=1-6 delims=:" %%a IN ( "%MY_START%:%MY_STOP%" ) DO (
     set H1=%%a
     set M1=%%b
     set S1=%%c
     set H2=%%d
     set M2=%%e
     set S2=%%f
     echo "%H2% %M2% %S2% - %H1% %M1% %S1%"
   )

I'm going to stop right here... I ran this from the command line about a dozen times with different arguments, and it didn't work.  The idea is really simple, place the timestamp into a string of colon delimited tokens and extract each of the fields and assign them sequentially to variables starting with %a% through %f% (six fields).  If I gave it the same arguments twice in a row it would be wrong the first time, and right each subsequent time.  If I ran from the command line the arguments would not be set.

Once I understood what was going on it was easy enough to fix.  Moving my echo outside of the loop caused it to behave properly... it took about an hour to finish up the script that I thought would take five minutes because of this behavior.  Convinced that there was a pointless bug in Dosshell I went off to consult the inter-webs to find out the logic behind this, what I discovered is a bit disturbing.

Dos Shell has something called "Delayed vs. Immediate Variable Expansion".  The default behavior is for the Shell interpreter to read in all of the lines between "(" and ")" at once and apply any variable expansion prior to executing any of these lines.  So if you are doing an IF/ELSE construct and have a series of assignments they will be expanded prior to evaluating the first expression:

   IF %H1% GTR %H2% (
      REM we have had a clock rollover since we started, add 24 hours
      set /a TEMP=%H2% + 24
      set /a HOUR2=%TEMP% * 3600
   )

In this example the TEMP variable will not be equal to "%H2% + 24", it will either be set to a previous value or not at all.  The disturbing thing is that Microsoft considers this a "feature" and not a "bug"; the documentation gives an example of how you could use a value stored in a variable and reset it to the original value all in one fell swoop... but since the value would be the same for both actions I don't see how this could ever be useful, or even do what they are saying cleanly.  The good news is that there are work-a-rounds... the bad news is that they are ugly.  The simplest method is to enable delayed expansion by setting an environment variable and then use alternate variable syntax (!VARIABLE!) in the instances where you want delayed expansion.  For me, I decided to leave my code as-is, and took a silent vow to avoid using Dos Shell in the future if any other alternative existed.

For completeness the code for the time delta is included below:

   @ECHO OFF
   REM > time_delta.bat <start> <stop>
   REM > time_delta.bat 2:03:04 2:05:06
   REM 122

   set MY_START=%1%
   set MY_STOP=%2%

   REM grab the hours, minutes and seconds from the start time fields 1 through 3
   REM grab the hours, minutes and seconds from the stop time  fields 4 through 6
   FOR /f "tokens=1-6 delims=:" %%a IN ( "%MY_START%:%MY_STOP%" ) DO (
     set H1=%%a
     set M1=%%b
     set S1=%%c
     set H2=%%d
     set M2=%%e
     set S2=%%f
   )

   REM convert the start time into raw seconds
   set /a MIN1=%M1% * 60
   set /a HOUR1=%H1% * 3600
   set /a TOTAL_1=%HOUR1% + %MIN1% + %S1%

   REM convert the stop time into raw seconds
   REM NOTE: the next line needs to be outside of the IF block in order to work due to
   REM       immediate variable expansion - variables are expanded when read, but the
   REM       commands are not executed until the ending brace is reached ")".  There
   REM       are other work-arounds for this, but none of them are clean.
   set /a TEMP=%H2% + 24

   IF %H1% GTR %H2% (
      REM we have had a clock rollover since we started, add 24 hours
      set /a MIN2=%M2% * 60
      set /a HOUR2=%TEMP% * 3600
   ) ELSE (
      REM no clock rollover, so convert directly into seconds
      set /a MIN2=%M2% * 60
      set /a HOUR2=%H2% * 3600
   )

   set /a TOTAL_2=%HOUR2% + %MIN2% + %S2%
   set /a DELTA=%TOTAL_2% - %TOTAL_1%
   echo %DELTA%

WHY I COMMENT MY CODE

I was reading an article (http://ardalis.com/when-to-comment-your-code) and it got me thinking about the subject, which is near and dear to my heart.

Over the years I've worked on a lot of projects, some for only a few days others spread out over years.  I've produced a lot of helper programs, implemented huge honking scripts and programs in a wide variety of languages over a wide range of platforms.  I've even managed large chunks of code in legacy systems spanning thousands of files.  I've worked in C, Objective-C, C++, Java, SQL, Bash, Ksh, Shell, Batch, Visual Basic, Tcl/Tk, Clearcase triggers, etc...  I've worked on Windows NT through Windows 7, Several Flavors of Linux including ones with Real-Time, Solaris, HP-UX, Powermax, Powerhawk, Lynx, some older flavors of Macintosh, VXWorks, etc...

I remember a day when a coworker was modifying a script to run on a new platform and asked me for some help.  It took a few minutes to figure out what the script was doing, I saw a few places for improvement, figured out the bug she was running into  and work out a solution.  As I was reading through the code I commented that it was a good script and she should consider sharing it with the department to which she responded "You do know you wrote this script, don't you?"  It occured to me that if it wasn't already in my coding style it would have been very difficult for me to figure out what the script was doing.  It was truly unfair to have created this huge program and then expect someone else to maintain it after I had lost interest (usually within a day or so).  I made sure that I left the program well commented before handing it back.

The idea that well-written code doesn't need comments is laughable.  This might fly if you are writing a tiny program, but once you start a project of any real scale - a few hundred thousand lines of code, written in various languages, various coding styles, maintained over a few decades by different companies... Even if you are writing a small program that someone else is going to use, and probably modify a few years down the line when your favorite language is no longer the defacto standard around the office and an intern needs to make a change... It is your duty to make sure that the code is well documented with sprinkled comments.

I consider comments to be bread crumbs that will aid a developer in understanding my code.  I like to strictly follow a coding standard; it doesn't really matter too much which one, but you should follow a consistent coding standard.  If you don't have one I like to do the following:
   - each file should have a header section that describes what the file is doing at a high level.
      //    - provide a revision history section, each entry is on one line
      // REVISION HISTORY
      // DSM  3Dec2012 added support for new signal types including COUNTS and 2-Byte floats
      // DSM  7Dec2012 moved signal information to a seperate class
   - provide a list of features you would like to add at some point, along with priority for implementation
      // TO-DO LIST
      // HIGH   move initialization data to an external file
      // LOW    move printing functionality to a seperate class
   - functions should contain header information that describe what the function is going to do
   - code should frequently comment what you are trying to do (at a high level)
   - if you are not using a tool to do versioning of your code you should provide change tracking in your code (which lines were touched by change number 5?)

If your compile environment supports any kind of documentation you should be using that as well... so for Visual Studio you should be using the XML style documentation that will allow for tool-tips when you mouse over a class or variable.

A lot of people dislike comments because they feel they can lie.  This is a maintenance issue, and when discovered the comments should be updated appropriately.  This is not a reason to do away with the comments entirely.  If anything it's a call for following better coding standards on updating comments.

It doesn't matter how clean you think your code is, some day an intern will attempt to make a change and if you don't give him a few breadcrumbs to follow he will fail.  When he fails it will not be his fault for not understanding your idiosyncratic coding style.  It will be your fault for not living up to the unspoken contract all paid developers have with their employers to write good and maintainable code.

Look at my coding samples for further examples of the minimum level of coding that should be acceptable.  Your code needs to be maintainable.

Finding Your Passion

During commencement addresses we often get the advice to “Follow your Passions” in order to be successful.  On the surface this sounds like good, sage advice; But, many people have no clue as to what their passions may be, or even how to discover them. 

Recently, I heard an interview on NPR where a high school student started down this path in order to figure out what he wanted to do with his life.  He contacted leading academics and ended up doing lunch with several important economists in order to find out “what dreams should I follow, if I have no dreams of my own.”  This got me to thinking about what kind of advice I would offer if someone approached me.  The interview was very dissatisfying; the economists asked a lot of questions but ended up offering no real advice except to follow the money.

In life it has been my experience that many people are not introspective.  Everyone is capable of being happy, everyone has experienced joy in their lives; but if you ask people “What do you enjoy doing?” many cannot find an answer.  I wouldn’t say this is bad, but without understanding yourself better you are denying yourself the ability to pursue happiness.  Passion is the same thing; I believe we all have the capability of becoming passionate about something, but discovering that passion can be difficult.

As a starting point to introspection I recommend something along the lines of the Rokeach Values, the idea is very simple: print out the 18 terminal values (below), cut them into separate cards and place them on a table.  Move them around to identify which ones you consider important, and which ones you consider less important.  Once you have selected your “important” pile you will start moving them around and sort them into order from most important (to you) to least important.  Then you will do the same thing with the 18 Instrumental values (also below).

This is the first step to finding your passion.  The terminal values represent how you would like life to turn out; when you die what would you like to have accomplished.  The instrumental values represent the path you would like to take to get there; Would you like to be acknowledged by your peers for your imagination? Your Helpfulness?  Your Intellect?  With this information under your belt you can start to look into fields that will help you to fulfill your goals.

My next piece of advice would be to pursue a Bachelor of Arts (which forces you to take a lot of non-core classes) as opposed to a more focused degree, and to start getting involved in lots of different activities.  Most activities will probably fail to hold your interest for more than a week or so, but that is good.  The goal is to expose yourself to lots of different experiences and viewpoints and… stuff.   By doing this you will start to uncover what you enjoy, what you hate, what drives you.  And you will start to uncover who you really are.

Along the way you will, hopefully, discover your passion.  And if you don’t, then you can follow the money.

http://www.npr.org/blogs/money/.../i-know-im-supposed-to-follow-my-passion-but-what-if-i-dont-have-a-passion

Design - Color Theory

Color Terminology

I am fascinated by User Interface design.  Over the years I have developed web pages and applications, I’ve done some rudimentary Graphic Design to create some custom backgrounds or logos.  When creating a design I like to think it through for usability, simplicity and elegance; and I have found that it is the little details that make the difference between an award-winning website and something that looks like a high-school student threw it together for an assignment.  As such, I find Color Theory a fundamental skill; and not just for Website Design or Interface Design, but as a transferable skill that can be used for presentations and gaining a deeper understanding of human psychology and physiology at some level.

To start colors have subtle, yet distinct meanings; the same way that two synonyms have different flavors.  When picking a color you should think about the kind of message you want to send:

Black – strength, professional, definite
Gray – balanced, calm
Pink – romantic, feminine, friendship, affection
Red – exciting, bold, youthful, danger
Orange – cheerful, friendly, confident, adaptability
Yellow – optimism, clarity, warmth, concentration
Green – natural, organic, calming, money
Blue – calming, dependable, clean, sincerity
Violet – loyalty, wisdom
Purple – creative, imaginative, tension, wealth

Or possibly your target audience:
Men’s Favorite Colors – Blue, Green.
Women’s Favorite Colors – Blue, Purple, Green, Red.

Colors are generally viewed two different ways: Additive – created by the mixing of light (adding red, green and blue together gives us white light), and Subtractive – created by the mixing of paints and objects (mixing cyan, magenta and yellow together gives us black, subtracting each color gets us back to white).  Additive colors (RGB) are used to describe colors on a computer monitor or website.  Subtractive colors are used to describe real world physical objects such as a poster.

If we were to set up a color-wheel with red at the top (lets’ call it 12-O’Clock or 0° position) and travelling through our rainbow through Green, then Blue and back again to Red we would see that Red, Green and Blue are equidistant from each other (0°, 120° and 240°).  These are often known as our Primary Colors and can be combined on your monitor to make white; in all honesty any three equidistant colors would work here but we like these because they line up more naturally with the way our eyes perceive colors.  Cyan, Magenta and Yellow are often referred to as Secondary colors; they can be combined to give us black ink and were chosen by the printing industry due to dye availability and cost.  Other colors are often referred to as Tertiary colors.

Warmer colors (the top half of our wheel) feel as if they are moving toward you, Cooler colors (the bottom half) feel as if they are receding.  Warmer colors work well for foreground objects while Darker colors work better for the background.  As an interesting side note there are certain types of 3D glasses that can amplify this effect and work right-out-of-the-box with many cartoons penned by traditional artists since they use rudimentary Color Theory when drawing.

RGB vs. HSV

Computers understand RGB (how much Red, Green and Blue to use to make our color); the computer usually stores these as hex values (a range from 0% to 100% becomes 0x00 through 0xFF (0123456789ABCDEF)).  Two Hex digits are used for each color (0xRRGGBB).  Color Theory works off of Hue, Saturation and Value though.  Let’s look at an example - the 216 Web-Safe colors come from creating all of the different combinations of 0%, 20%, 40%, 60%, 80%, 100%) values:

    Decimal   Hex      Percent
    0         0x00     0%
    51        0x33     20%
    102       0x66     40%
    153       0x99     60%
    204       0xCC     80%
    255       0xFF     100%

As you can imagine, it is difficult to determine which color is 30° away from Red (0xFF0000 or #FF0000).  In order to determine which colors to look at we need to convert to HSV (Hue, Saturation and Value). 

Hue is defined as the perceptible color corresponding to a specific wavelength of light (eg. Bright Red Violet “#CC00FF” – Violet “#9933FF” – Blue Violet “#6633FF”).  This is normally represented as a value between 0° and 360°, we will treat Red as 0°, Green as 120° and Blue as 240°.

Value is either the Lightness or Darkness of a color; depending on the conversion formula this is referred to as Tint (combined with white), Tone (combined with white/black) or Shade (combined with Black).  The exact formula you use for conversions will cause some minor variations in much the same way that choosing a Black vs. a White Movie Theater screen will cause variations (eg. Red “#FF0000” – Carnation Pink “#FF99CC” – Empire Ruby Red “#CC3333” – Deep Burnt Sienna “#660000”).

Saturation (Chroma) is the relative intensity of a chromatic color, or how far away from Gray the color is (eg. Pure Cyan “#00FFFF” – Carribean Blue “#33CCCC” – Mediteranian Blue “##339999” – Deep Blue-Green “#336666” – Charcoal Gray “#333333”).  A color is considered pure Gray-Scale if Red==Green==Blue.

So, to translate we are going to switch to some pseudo-code:
   RGB_to_HSV()
   {
      Min = min(R, G, B)
      Max = max(R, G, B)
      Value = Max         // this is our brightness
      Delta = Max – Min
      If (Max != 0)
         Saturation = Delta / Max
         If (R == Max)      Hue = (G – B)/Delta
         Else If (G == Max) Hue = 2 + (B – R)/Delta
         Else               Hue = 4 + (R – G)/Delta
         END_IF

         Hue *= 60
         If (Hue < 0) Hue += 360
         END_IF
      Else
         Saturation = 0
         Hue = -1
      END_IF
   }

Basically we are using Brightness for our Value (how close are we to white).  We could also add our R + G + B together here and get slightly different results.  It can be fun tweaking these formula to see how the results change.  Hue gives us a value between 0° and 360°; If Red is our predominant color we get a value between -60° and 60°.  Saturation is a measure of how closely the color is to Gray-Scale or how closely the three colors match in intensity.

Now that we have our Hue values we can start talking about color combinations:
Picking three colors that are side-by-side on the Color Wheel (eg. Yellow, Spring and Green at 0°, +30°, -30°) provides a harmonious balance, these are called Analogous colors.  Analogous colors are relaxing and grounding.  Neutral combinations are a little more closely related so more subtle, these are 0°, +15° and -15°.

Complimentary colors on the other hand represent a contrast, they provide energy with some discord; they are located directly across from each other on the wheel (0°, +180°)..  Split-Complimentary colors are a little easier to work with (0°, +150°, -150°) and open up a third color choice.

If you were feeling more confident you could try a Triadic Combination (0°, +120°, -120°) which provides a lot of tension.  Tetradic (0°, +90°, +180°, -90°) or even 4-Tone (0°, +60°, +180°, +240°).

Tonal Color Harmony can be obtained by adjusting the base colors Tint, Tone or Shade.  Chromatic Color Harmony is based on adjusting the Hue and Brightness while keeping the Saturation the same.

Monochromatic Color Harmony can be achieved by playing with the Hue, Tint, Tone, and Shade individually.

Summary

So, what does all of this actually mean?  If you want to design something (presentation, webpage, software, painting, etc…) you should try to pick a color that enhances your message for your base color.  After  you have picked that color you should choose a few more colors to round out your pallete.  It is best to limit yourself to four or fewer colors;  I’d recommend Analogous or Split-Complimentary colors unless you are feeling extra bold.

With this information under your belt you should be ready to create an attractive looking… whatever.

And now for some links to sites that probably describe color theory far better than I just did:
http://www.worqx.com/color/
http://www.colorvoodoo.com/