|Problems? Is your data what you think it is?|
simon.proctor's scratchpadby simon.proctor (Vicar)
|on Jun 02, 2004 at 09:08 UTC||Need Help??|
Trying to convince co-workers on testing, refactoring and simple design methods I could do with some feedback on this. Many parts pinched from Code Complete and Refactoring:
Design, Implementation and Refactoring
Design is considered a 'wicked problem'. In many cases, to produce a design the problem is solved twice. To produce the design, the problem is solved (even if only in part) and then solved again to prove that the solution works. In fact, it may be that only once the problem is solved that side problems emerge. Their existance proving to be unknown until the original problem had been worked on.
Whatever the process, a design is produced via a combination of heuristic judgements, best guesses and assumptions. Many mistakes are made in the process of the design because of this. In fact, a good solution and a sloppy one may differ only in one or two key decisions or perhaps choosing the right tradeoffs.
Because this, good designs evolve through meetings, discussions and experience. In some cases, they also improve through partial implementation (hence the 'wicked problem' moniker).
A design, to stand a chance of working, should also restrict possibilities. Because time and resources are not infinite the goal of the design should be to simplify the problem into an acceptable form for implementation. Not all processes for this are the same. Each new problem introduces an entirely new set of variables. Failure to recognise this can result in the wrong technique, tool or process being applied.
The success of the implementation can be measured in different ways. Glibly, it can be measured as 'it does what we want' and is then left to rot until the next problem occurs. Many projects associated with problems fail due to poor management, requirements (etc) but equally many (especially software projects) fail due to complexity.
Managing complexity is a key factor in ensuring success. If a solution is too complex (either by design or evolution) then it becomes increasingly impossible to maintain. This is a major source of cost and resource overhead.
Complexity can arise in these simple cases (say):
A complex solution to a simple problem
A simple, incorrect solution to a complex problem
An inappropriate, complex solution to a complex problem
Managing this allows many design considerations to become much more straightforward.
Characteristics of a good design:
Ease of maintenance
High use of low level utility (software design)
Low level fan out (software design)
Layered (predominantly software design)
Over time, common solutions to common problems emerge. These common solutions are reasonably abstract. Enough to be applicable in a general sense but specific enough that they can be recognisably applied to the solution. Application of common solutions (or patterns) can achieve many of the above characteristics. Unfortunately, they are not always applied (or worse applied incorrectly) because of the reasons already stated.
It is often the case that the first implementation (or even the first few of many) are not easily maintainable, simple or reusable. Rather than stay stuck in the design process, it can and is more advantageous to take a pragmatic approach. As stated, it can be impossible to completely solve a problem satisfactorily without first solving it.
Here, the best approach is to attempt to make the best decisions possible at the time. Then, re-examine the problem and solution for signs of a good and/or bad design. With the experience of the implementation, improvements can be easier to spot and mistakes easier to find. This process is called refactoring. Refactoring can include shifting to patterns or between patterns where applicable, removing now uneeded functionality or reducing complexity.
By refactoring, we examine what we thought we knew, what we tried and what actually happened and try to make it
simpler easier to maintain
reusable if possible
easier to understand
Even achieving only one of these can be critical to the long term success of a project.
While implementing, analysing and refactoring a solution to a problem, it is important to be able to prove your solution works as promised. Critically, it is also important to be able to proce what happens to your solution when things don't go to plan. Understanding your corner cases, the limits of your inputs and outputs will test your assumptions. It will also provide for planning for and mitigating unforseen circumstances (at least as much as possible).
In software design, software testing is used to provide this metric. By tieing the development of tests directly to the implementation of the software solution, the solution is built in parallel to the tests that prove the solution works. Aside to the benefits above, this testing also provides
a test of the design at a low level (how it works, couples, simplicity etc)
proof that changing one part of the system hasn't broken another part of the same system
Accepting that software must evolve as requirements change and the complexity of the solution changes mandates that software testing be included in the production of any solution right from the outset. This testing allows the assumptions to be checked and rechecked even before higher level testing is considered. After all, if the software doesn't work as advertised there is no point arranging usability and acceptance testing. If you don't know how your software will fail there is no point putting it up for client review.
In producing your implementation, it is also important to code defensively. Rather than assume that resources are available (say) you test your assumptions are true before working with them. By doing so, you produce a simple first candidate area for your software tests. If you are working with a resource and it 'goes away' you can produce a test that re-enacts this scenario. Once this test is written, you work with your code until all the problems are fixed, handled appropriately or are documented. In this simple process of defensive programming coupled with testing, the reliability of the solution should dramatically increase.
Testing can also form part of the installation process. Deployment of the solution needs proof that it is installed and operating correctly. As a suite of tests and benchmarks have already been produced, what better method of proving the deployment is ready for acceptance testing?
Examine the problem simply
Worry about only what you need to implement
Implement it as best you can
Rexamine the solution and the problem
This can be helped with defensive programming, pragmatic design, refactoring to common solutions where appropriate and testing at all stages of implementation.
Here is the problem on the prod. machine
They really screwed this machine as it only has the 64bit libs so you can't do ordinary fire and forget compilation :).
My home machine throws a similar error. However, it looks like that perl (5.61) was built with cc. The prod one was with gcc. However, I am tempted to uninstall the package installations and build from scratch. At least then I'll know if my compiler is setup ok! Bit risky though!
Here is the output of the prod. perl -V:
Skeleton code for receiving unbuffered keyboard io on windows
Please let me know what you think as I want to use it as a basis for a variety of new console apps I am to write. It should compile in a new VStudio project directly. Feel free to poke it at a fellow developer. I had something like this ages ago but without CVS (or VSS *shudder*) the code got *ahem* mislaid.
Function pointer despatch table
If I wanted to move this to a despatch table of arbitrary size would I use malloc and pointers to pointers? I say pointers to pointers as I was wondering about how to make the size of the despatch table dynamic. So I don't hardcode the size in the function definition/prototype. I can pass it as a parameter instead.
Thoughts for CV (this is for your eyes rather than actual CV speak):
Work stuff Since the last edit to my CV, the Java stuff, I have concentrated on two projects at work. The first is the HR Project management application. This application handles all project initialisation for HR managers. It does this by capturing form information, storing in SQL Server and then transmitting to authorisers and delegates via email and pdf. The user is guided through an authorisation and approval process automatically.
There is more to this project, delegation history, reports, generation of PDF via XSL (etc) but its more for talking about in person I think.
The other major project is the implementation of an Intranet system using .NET and a package called Sitecore. This is a pure .NET product that we are customising to use within the company. I have been tasked with importing legacy data from across all the old php/perl/asp intranets into the system. So far I have written a news importer tool that combines a desktop importer application with a webserver that sits on the intranet server (also written by me).
In general I support and maintain all the Unix/Linux services for the team, solve all the DNS and domain registrations/problems and advise on how better to do things. My support ticketing system is used daily by the whole team.
This application reads an exchange mail box for outlook task requests and stores them in an SQL Server database. My Perl web application then allows the team to log in, accept, assign and close these tickets. The reports from this application are being used in the monthly managers meetings.
I have a general interest in development and programming which I have progressed into some freelance work. Currently I have completed a Cottage booking website (Perl) and am in the process of completing a desktop kennel management program in .NET. I am also maintaining a collection of websites with a view to improving their search engine results.
The kennel management program allows the operator to add, view and edit clients and their animals from a windows based interface. They can search for these clients and animals and make bookings for them. These bookings are then printable in report format for filing and general office administration.
The project has involved creating a number of custom windows components including a wizard style interface that is used throughout the application. The reporting is provided via a purchased component library to keep within project deadlines and promote reuse of code.
I think this works. It was inspired by one of my SQL puzzle books. We know that we can get all the open ticket times but want the most recent close date for each row. I wanted to ensure that the open date got used only once and didn't realise I could nest selects. Here is the solution based on my test date:
SQL Genius? Or lucky ? Maybe a genius is simply someone who is always lucky :).
Assumes you use the brackets.
Here is the shuffle thing. The idea is that you may want to create a large shuffled list from a smaller source. If you simply shuffle the source then you get a repeated pattern in your list. So this one shuffles whenever you reach a pattern point. This point is when our index is a multiple of the size of the source. The array here is hardcoded (called @copy).
The latest thing. Guess how it makes me feel.
Service orientation is a means for building distributed systems. At its most abstract, service orientation views everything-from the mainframe application to the printer to the shipping dock clerk to the overnight delivery company-as a service provider. Service providers expose capabilities through interfaces. Service-oriented architecture maps these capabilities and interfaces so they can be orchestrated into processes. The service model is "fractal": the newly formed process is a service itself, exposing a new, aggregated capability.
this link for unbuffered io for my tail program?
You have an administrators table (ID, firstname), a privs table (ID, name) and a modules table (ID, module).
The challenge is to return all privs for all modules for a given user. However, if the user does not have a privilege then a result must still be returned but the flag set to zer.
for user simon:
So Simon has not privs on module test and one in bob. This is what I have so far
We can do away with adminID and moduleID at a later stage. Have fun!
The solution!! Taa daa!!
Free Nodelet freed
Perl and Address labels, oh yeah!
My Year in Perl, 2004 edition
Bringing Logic Programming to Perl
shared mem segments with Inline::C
Re: Printing multiple arrays as multiple column
Higher Order Perl Mailing List
Convert PowerPoint Presentation to Word Document with Win32::OLE
Finding Windows XP CD Key
Visualize packed data as bits
Tales from writing a RPN evaluator in Perl 5, Perl 6 and Haskell
Closed geometry: a train track problem
$picture eq ('word' x 1000)
Project Euler (a series of challenging mathematical/computer programming problems)
Re: Looking for help with AI::Genetic and classroom scheduling
A Beginning Guide To Evolutionary Algorithms
Operator Associativity and Eliminating Left-Recursion in Parse::RecDescent
:lvalue support in Object::InsideOut
Subroutine attributes for solving crosscutting problems
Evil Interview Questions