- IBM Federal Systems Division
As a Senior Associate Programmer working on a secret spy satellite program (long since declassified), I was assigned the task of taking over the program which processed telemetry from the Star Sensor (Orientation of the satellite was determined using navigation by the stars). I was told there were two main problems with the program which had been coded in the Fortran Language. 1. It was taking too long to achieve “lock” (identify a known sequence of “bright” stars based on date and time of day, and 2. It lost “lock” whenever the sun was in view of the star sensor. The first problem was easy. Management thought that by recoding the program in Assembler Language it would run faster. Actually, the Fortran compiler produces code which is optimized for speed.
What I immediately saw was that a simple iterative process of adding a set of brightness values together until the sum matched the brightness of the brightest star in the field of interest was executed over and over again, bumping the starting index by one then adding all the subsequent readings until the reading just past the previous set was added. Obviously (to me, anyway), the new sum could be achieved simply by subtracting the value at the previous first index from the previous sum, then adding the value at the next end index. This would reduce the number of additions required to process one telemetry buffer by a factor of 25 to 1. I recoded this part of the program in Assembler Language using the revised logic, and lo, the problem with taking too long to achieve “lock” went away.
The managers were still scratching their heads about the problem with losing lock every day when the Sun came into view. Rumor had it that there was a “sun filter” which would supposedly handle the problem, but nobody could figure out why it wasn’t working. I took the initiative to research the documentation provided by the Hardware Engineering Group, and although this reading can be very dry and non-interesting, I came across an appendix which listed all the possible commands which could be sent to the satellite. Listed among these commands was one called “deploy sun filter”.
Following the sage advice of Grace Hopper1, I found the place in the Star program where it was recognized that the Sun was in view of the star sensor. I coded the instructions which would cause the magic command to be sent to the Satellite, three days later, I was hunted down by the Program’s Chief Scientist who exclaimed that the satellite had been in continuous lock for three days, now, and he just had to meet the person who had ingeniously solved the problem with the Sun causing the satellite to lose lock.
- Grace Hopper famously advised her programmers: “it’s always easier to apologize than to ask permission”.
- Community Housing Resource Center (CHRC)
CHRC is a HUD-certified non-profit housing counseling center which relies on government grants and donations from private enterprises (banks, real estate businesses) which can profit from their services. In order to be eligible for government grant money, the center must have HUD-Certified counselors to do the counseling. In order to collect the government funds, reporting of counseling activities and detailed information about each client case must be provided in a timely manner.
The counselors record information in the HUD-approved Client Management System (CMS) each time they meet a client or do work on behalf of the client. The collection (extraction) and reporting of this data takes time away from the counselors. The reporting is done via computer records, which requires a technically-educated person with the ability to speak Computer.
When I started at CHRC, I observed that the current state of the Housing Counselor Online (HCO) CMS admin account indicated a need for closer attention being paid to program updates, and data reporting requirements. The counselors had been tasked with copying information about each Foreclosure Prevention client onto a 40-column spreadsheet whenever a new case was opened, then again when the case was closed. At the end of each month, this spreadsheet was sent to the parent organization Washington State Housing Finance Commission (WSHFC) as a billing voucher. WSHFC, in turn, uploaded the spreadsheet to the Neighborworks organization which was in charge of administering the National Foreclosure Mitigation Counseling (NFMC) program financed by HUD
I was astounded that no effort had been made to simply extract the information from the database, observing that a subset of the information was not currently recordable in that database. I also noted that the HCO introductory log-in page referred to an update which would include the extra data, but it requested a modification to the set-up portion of the administrator’s account. I took the initiative to make the required modification, then proceeded to fill in the extra data using the client’s paper files.
The HCO program then made available every parameter needed for the spreadsheet through an extraction process. The HCO development team did not, however, deem it necessary to present the extracted data in the order defined by the NFMC reporting requirements document, since the NFMC was expecting the data to be uploaded to them as a CSV (comma separated variables) file, which does not require column-specific placement of the data. CHRC was required by WSHFC to use the column-specific format. My experience with programming told me that this problem was solvable through software. I researched the currently available open-source programming languages for a solution, and came up with using the Python programming language, which has a well-defined module available for handling of CSV files, allowing the program to define the order in which the columns should be presented. I then wrote a program to read the file extracted from HCO, make a few modifications to the extracted data, and write out the data in the column order required by WSHFC. Monthly production of the billing voucher thus turned from a time-consuming task which had eaten literally hours of counselor time to a 15-minute exercise.
3. Oregon Opportunity Network (Oregon ON)
Oregon ON is a non-profit organization based in Portland, Oregon which serves as support for their member non-profits in the State of Oregon. The State of Oregon Housing and Community Services (HCS) in Salem had asked all the Housing Counseling agencies in the state to provide a quarterly report detailing information they collected about all their clients who were being counseled as Pre-purchase Homebuyers. Most of the agencies were using the CMS named Counselor Max which is accessible by monthly subscription. Three of the agencies were using the free HCO program. Oregon ON arranged for the Counselor Max users to be trained for identifying the data items to be presented on the report. Since HCO is a free program, the parent company does not offer in-person training.
As a recognized expert in the use of HCO (Home Counselor Online), I was asked to present a training session for the agencies in Oregon who use this database. The object was to identify parameters needed for the new report required by the State of Oregon: (HCS), where to save the parameters in HCO, and how to extract the data. The HCO database is capable of storing most of the required data, but manipulation is necessary to convert some of the values to the requested format (e.g. The report asks for the Age of the client at intake, but this information is stored in HCO as two parameters (Date of Intake and Date of Birth.), and of course, when extracting data from a data management system, it is not possible to specify the order in which to present the data.
Using the csv module of Python, I wrote a program to generate the Oregon HCS report which uses the custom data file extracted from HCO, converts data which is not stored exactly in HCO to the requested format, and rewrites the data in the order specified by the Oregon HCS report data template, thus being able to match the data points to the Headers specified by HCS. This operation is commonly described as “Custom Report Enhancement”. The conversion of data from the HCO format to the HCS format could have been done by hand, but by using the computer program, the time to effect this operation is reduced to milliseconds rather than hours.