Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

simon.proctor's scratchpad

by simon.proctor (Vicar)
on Jun 02, 2004 at 09:08 UTC ( #359287=scratchpad: print w/ replies, xml ) Need Help??

Trying to convince co-workers on testing, refactoring and simple design methods I could do with some feedback on this. Many parts pinched from Code Complete and Refactoring:
Design, Implementation and Refactoring

Design is considered a 'wicked problem'. In many cases, to produce a design the problem is solved twice. To produce the design, the problem is solved (even if only in part) and then solved again to prove that the solution works. In fact, it may be that only once the problem is solved that side problems emerge. Their existance proving to be unknown until the original problem had been worked on.

Whatever the process, a design is produced via a combination of heuristic judgements, best guesses and assumptions. Many mistakes are made in the process of the design because of this. In fact, a good solution and a sloppy one may differ only in one or two key decisions or perhaps choosing the right tradeoffs.

Because this, good designs evolve through meetings, discussions and experience. In some cases, they also improve through partial implementation (hence the 'wicked problem' moniker).

A design, to stand a chance of working, should also restrict possibilities. Because time and resources are not infinite the goal of the design should be to simplify the problem into an acceptable form for implementation. Not all processes for this are the same. Each new problem introduces an entirely new set of variables. Failure to recognise this can result in the wrong technique, tool or process being applied.

The success of the implementation can be measured in different ways. Glibly, it can be measured as 'it does what we want' and is then left to rot until the next problem occurs. Many projects associated with problems fail due to poor management, requirements (etc) but equally many (especially software projects) fail due to complexity.

Managing complexity is a key factor in ensuring success. If a solution is too complex (either by design or evolution) then it becomes increasingly impossible to maintain. This is a major source of cost and resource overhead.

Complexity can arise in these simple cases (say):

A complex solution to a simple problem
A simple, incorrect solution to a complex problem
An inappropriate, complex solution to a complex problem

Managing this allows many design considerations to become much more straightforward.

Characteristics of a good design:

Minimal complexity
Ease of maintenance
Loose coupling
Extensibility
Reusability
High use of low level utility (software design)
Low level fan out (software design)
Portable
Lean
Layered (predominantly software design)
Standard techniques

Over time, common solutions to common problems emerge. These common solutions are reasonably abstract. Enough to be applicable in a general sense but specific enough that they can be recognisably applied to the solution. Application of common solutions (or patterns) can achieve many of the above characteristics. Unfortunately, they are not always applied (or worse applied incorrectly) because of the reasons already stated.

It is often the case that the first implementation (or even the first few of many) are not easily maintainable, simple or reusable. Rather than stay stuck in the design process, it can and is more advantageous to take a pragmatic approach. As stated, it can be impossible to completely solve a problem satisfactorily without first solving it.

Here, the best approach is to attempt to make the best decisions possible at the time. Then, re-examine the problem and solution for signs of a good and/or bad design. With the experience of the implementation, improvements can be easier to spot and mistakes easier to find. This process is called refactoring. Refactoring can include shifting to patterns or between patterns where applicable, removing now uneeded functionality or reducing complexity.

By refactoring, we examine what we thought we knew, what we tried and what actually happened and try to make it

simpler easier to maintain
reusable if possible
easier to understand

Even achieving only one of these can be critical to the long term success of a project.

While implementing, analysing and refactoring a solution to a problem, it is important to be able to prove your solution works as promised. Critically, it is also important to be able to proce what happens to your solution when things don't go to plan. Understanding your corner cases, the limits of your inputs and outputs will test your assumptions. It will also provide for planning for and mitigating unforseen circumstances (at least as much as possible).

In software design, software testing is used to provide this metric. By tieing the development of tests directly to the implementation of the software solution, the solution is built in parallel to the tests that prove the solution works. Aside to the benefits above, this testing also provides

a test of the design at a low level (how it works, couples, simplicity etc)

proof that changing one part of the system hasn't broken another part of the same system

Accepting that software must evolve as requirements change and the complexity of the solution changes mandates that software testing be included in the production of any solution right from the outset. This testing allows the assumptions to be checked and rechecked even before higher level testing is considered. After all, if the software doesn't work as advertised there is no point arranging usability and acceptance testing. If you don't know how your software will fail there is no point putting it up for client review.

In producing your implementation, it is also important to code defensively. Rather than assume that resources are available (say) you test your assumptions are true before working with them. By doing so, you produce a simple first candidate area for your software tests. If you are working with a resource and it 'goes away' you can produce a test that re-enacts this scenario. Once this test is written, you work with your code until all the problems are fixed, handled appropriately or are documented. In this simple process of defensive programming coupled with testing, the reliability of the solution should dramatically increase.

Testing can also form part of the installation process. Deployment of the solution needs proof that it is installed and operating correctly. As a suite of tests and benchmarks have already been produced, what better method of proving the deployment is ready for acceptance testing?

Summary

Examine the problem simply
Worry about only what you need to implement
Implement it as best you can
Rexamine the solution and the problem
Repeat

This can be helped with defensive programming, pragmatic design, refactoring to common solutions where appropriate and testing at all stages of implementation.
Here is the problem on the prod. machine
gcc -B/usr/ccs/bin/ -c -fno-strict-aliasing -D_LARGEFILE_SOURCE -D_F +ILE_OFFSET_BITS=64 -O -DVERSION=\"3.14\" -DXS_VERSION=\"3.14\" -fPI +C "-I/usr/local/lib/perl5/5.8.0/sun4-solaris/CORE" Cwd.c In file included from /usr/local/lib/perl5/5.8.0/sun4-solaris/CORE/per +l.h:2686, from Cwd.xs:2: /usr/local/lib/gcc-lib/sparc-sun-solaris2.8/3.2.2/include/math.h:25:26 +: iso/math_iso.h: No such file or directory make: *** [Cwd.o] Error 1
They really screwed this machine as it only has the 64bit libs so you can't do ordinary fire and forget compilation :).

My home machine throws a similar error. However, it looks like that perl (5.61) was built with cc. The prod one was with gcc. However, I am tempted to uninstall the package installations and build from scratch. At least then I'll know if my compiler is setup ok! Bit risky though!

Here is the output of the prod. perl -V:
Summary of my perl5 (revision 5.0 version 8 subversion 0) configuratio +n: Platform: osname=solaris, osvers=2.8, archname=sun4-solaris uname='sunos solaris 5.8 generic_108528-11 sun4u sparc sunw,ultra- +5_10 ' config_args='-Dcc=gcc -B/usr/ccs/bin/' hint=recommended, useposix=true, d_sigaction=define usethreads=undef use5005threads=undef useithreads=undef usemultipl +icity=undef useperlio=define d_sfio=undef uselargefiles=define usesocks=undef use64bitint=undef use64bitall=undef uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='gcc -B/usr/ccs/bin/', ccflags ='-fno-strict-aliasing -D_LARGEF +ILE_SOURCE -D_FILE_OFFSET_BITS=64', optimize='-O', cppflags='-fno-strict-aliasing' ccversion='', gccversion='3.1', gccosandvers='solaris2.8' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=4321 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=1 +6 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='off_t', + lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='gcc -B/usr/ccs/bin/', ldflags =' -L/usr/local/lib ' libpth=/usr/local/lib /usr/lib /usr/ccs/lib libs=-lsocket -lnsl -lgdbm -ldl -lm -lc perllibs=-lsocket -lnsl -ldl -lm -lc libc=/lib/libc.so, so=so, useshrplib=false, libperl=libperl.a gnulibc_version='' Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' ' cccdlflags='-fPIC', lddlflags='-G -L/usr/local/lib' Characteristics of this binary (from libperl): Compile-time options: USE_LARGE_FILES Built under solaris Compiled at Jul 22 2002 02:55:19 @INC: /usr/local/lib/perl5/5.8.0/sun4-solaris /usr/local/lib/perl5/5.8.0 /usr/local/lib/perl5/site_perl/5.8.0/sun4-solaris /usr/local/lib/perl5/site_perl/5.8.0 /usr/local/lib/perl5/site_perl .

Skeleton code for receiving unbuffered keyboard io on windows

Please let me know what you think as I want to use it as a basis for a variety of new console apps I am to write. It should compile in a new VStudio project directly. Feel free to poke it at a fellow developer. I had something like this ages ago but without CVS (or VSS *shudder*) the code got *ahem* mislaid.
#include <stdio.h> #include <windows.h> #define SLEEP 100 /* Function prototype for the ctrl signal handler */ BOOL CtrlHandler( DWORD fdwCtrlType ); /* Currently global so we can set it in our ctrl-c handler to cleanly exit */ int exitflag = 0; int main() { HANDLE std_in; DWORD records_read = 0; DWORD i; LPDWORD old_mode; INPUT_RECORD buffer[128]; int sleep_time; /* Number of milliseconds to sleep */ sleep_time = SLEEP; /* Assign a handle to stdinput */ std_in = GetStdHandle(STD_INPUT_HANDLE); if (std_in == INVALID_HANDLE_VALUE) { printf("Could not get handle to stdin\n"); exit(EXIT_FAILURE); } /* Set ourselves able to receive input without a carriage return * +/ /* Backup the old mode */ if (! GetConsoleMode(std_in, &old_mode) ) { printf("Could not save old screen mode\n"); exit(EXIT_FAILURE); } /* We want to handle ctrl-c and disable all else */ SetConsoleMode(std_in, ENABLE_PROCESSED_INPUT); /* Register for ctrl-c */ if( SetConsoleCtrlHandler( (PHANDLER_ROUTINE) CtrlHandler, TRUE ) +) { /* Inifite loop - exits via the ctrl-c handler (way below) */ while(!exitflag) { GetNumberOfConsoleInputEvents(std_in, &records_read); if( records_read > 0) { /* We have events */ if( ReadConsoleInput(std_in, buffer, 128,&records_read +) ) { for (i = 0; i < records_read; i++) { /* Do something with buffer[i] - print in this + case */ if(buffer[i].Event.KeyEvent.bKeyDown) { if(buffer[i].Event.KeyEvent.wVirtualScanCo +de == 0x1c) { printf("\n"); } else { char c = buffer[i].Event.KeyEvent.uCha +r.AsciiChar; printf("%c", c); } } } } FlushConsoleInputBuffer(std_in); } else { } /* Preserve our cpu.*/ Sleep(sleep_time); } /* Do our shutdown processing here if need be. */ } else { printf( "\nERROR: Could not set control handler"); } /* Restore the old mode */ SetConsoleMode(std_in, old_mode); return 0; } /* Handle a ctrl event. Coded to handle ctrl-c only. */ BOOL CtrlHandler( DWORD fdwCtrlType ) { switch( fdwCtrlType ) { // Handle the CTRL-C signal. case CTRL_C_EVENT: /* We want to exit cleanly */ exitflag = 1; return( TRUE ); default: return FALSE; } }

Function pointer despatch table

If I wanted to move this to a despatch table of arbitrary size would I use malloc and pointers to pointers? I say pointers to pointers as I was wondering about how to make the size of the despatch table dynamic. So I don't hardcode the size in the function definition/prototype. I can pass it as a parameter instead.

#include <stdio.h> /* General program actions. */ void func_one(); void func_two(); void func_three(); /* Sets up the despatch table */ voi d setup( void (*despatch[3])()); int main() { /* Declare and setup the despatch table */ void (*despatch[3])(void) = {NULL}; setup(despatch); /* Call the second program action */ (*despatch[1])(); return 0; } void setup( void (*despatch[3])()) { despatch[0] = &func_one; despatch[1] = &func_two; despatch[2] = &func_three; } void func_one() { printf("Program action one\n"); } void func_two() { printf("Program action two\n"); } void func_three() { printf("Program action three\n"); }

Thoughts for CV (this is for your eyes rather than actual CV speak):

Work stuff Since the last edit to my CV, the Java stuff, I have concentrated on two projects at work. The first is the HR Project management application. This application handles all project initialisation for HR managers. It does this by capturing form information, storing in SQL Server and then transmitting to authorisers and delegates via email and pdf. The user is guided through an authorisation and approval process automatically.

A project begins by choosing an initialisation type. This sets out which forms the user will see. They may complete these forms in any order and only when all mandatory questions are complete may they then request authorisation. Additionally, some questions are only mandatory depending on an answer in the same form, the application also allows for this both at the server and at the client end (via Javascript).

There is more to this project, delegation history, reports, generation of PDF via XSL (etc) but its more for talking about in person I think.

The other major project is the implementation of an Intranet system using .NET and a package called Sitecore. This is a pure .NET product that we are customising to use within the company. I have been tasked with importing legacy data from across all the old php/perl/asp intranets into the system. So far I have written a news importer tool that combines a desktop importer application with a webserver that sits on the intranet server (also written by me).

In general I support and maintain all the Unix/Linux services for the team, solve all the DNS and domain registrations/problems and advise on how better to do things. My support ticketing system is used daily by the whole team.

This application reads an exchange mail box for outlook task requests and stores them in an SQL Server database. My Perl web application then allows the team to log in, accept, assign and close these tickets. The reports from this application are being used in the monthly managers meetings.

Home stuff

I have a general interest in development and programming which I have progressed into some freelance work. Currently I have completed a Cottage booking website (Perl) and am in the process of completing a desktop kennel management program in .NET. I am also maintaining a collection of websites with a view to improving their search engine results.

The kennel management program allows the operator to add, view and edit clients and their animals from a windows based interface. They can search for these clients and animals and make bookings for them. These bookings are then printable in report format for filing and general office administration.

The project has involved creating a number of custom windows components including a wizard style interface that is used throughout the application. The reporting is provided via a purchased component library to keep within project deadlines and promote reuse of code.
I think this works. It was inspired by one of my SQL puzzle books. We know that we can get all the open ticket times but want the most recent close date for each row. I wanted to ensure that the open date got used only once and didn't realise I could nest selects. Here is the solution based on my test date:
select tsa1.ticket_ID, ts1.name, tsa1.date_updated as date_opened, (select min(date_updated) from TICKET_STATE_AUDIT where date_updat +ed >= tsa1.date_updated and ticket_ID = tsa1.ticket_ID and state_ID = + (select ID from TICKET_STATE where name = 'closed')) from TICKET_STATE_AUDIT as tsa1 left join TICKET_STATE as ts1 on tsa1.state_ID = ts1.ID where tsa1.ticket_ID = 4 and tsa1.state_ID = (select ID from TICKET_STATE where name = 'open')
SQL Genius? Or lucky ? Maybe a genius is simply someone who is always lucky :).


Assumes you use the brackets.
use strict; my $input = "[http://forum1.reith.bbc.co.uk/cgi-perl/h2/h2.cgi|title]" +; my $nasty_input = "[http://\nforum1.\nreith.bbc.co.uk/cgi-perl\n/h2/h2 +.cgi|title]"; my $garbage = "blahblabhladas sdfgsdgf.\ndsfg\n\t\tsdfxsdf\n\tasf"; my $poo = join('', reverse( split(//, $garbage) )); my $final_input = $nasty_input . $garbage . $input . $poo . $garbage . + $poo . $input . $garbage . $nasty_input; print $final_input,"\n"; # This assumes that the delimiters surround our urls. # The next step would be to pass $string to something like URI to test + if it # is a valid url. Either that or to URI::Find to see if there is a sub + url in there. # Though I think URI::Find wouldn't like the |title stuff on the end. while($final_input =~ /\[(.*?)\]/sgc) { my $string = $1; if($string =~ /\n\r?/sg) { $string =~ s/\n\r?//sg; } print $string,"\n"; }
Here is the shuffle thing. The idea is that you may want to create a large shuffled list from a smaller source. If you simply shuffle the source then you get a repeated pattern in your list. So this one shuffles whenever you reach a pattern point. This point is when our index is a multiple of the size of the source. The array here is hardcoded (called @copy).
use strict; use Win32; use Win32::Console::ANSI; use Term::ANSIColor; my @colours = ( 'bold blue', 'bold blink yellow', 'red', 'white', 'green', 'cyan', 'magenta', 'bold red' ); my $list = make_list(12, @colours); #print join("\t", @{$list}); print "\n"; foreach my $colour (@{$list}) { print colored ("*", $colours[$colour]); print "\r"; sleep(1); } sub make_list { my ($size, @cols) = @_; unless(defined($size) || $size <= 0) { $size = $#cols; } # We need to grab all the possible numbers from # our colours, shuffle those and then pick the # first 'size' list. To avoid having a repeated # pattern, we reshuffle the list after we reach # a potential period point. my @copy = qw(0 1 2 3 4 5 6 7); shuffle(\@copy); my @array; my $j = 0; for(my $i = 0; $i < $size; $i++) { push @array, $copy[$j]; $j++; if(($j % $#cols) == 0) { # Reshuffle $j = 0; @copy = qw(0 1 2 3 4 5 6 7); shuffle(\@copy); } } return \@array; } # The Fisher-Yates Shuffle - taken from node 30243 sub shuffle { my $arrayref = shift; my $i = @$arrayref; while ( $i-- ) { my $newloc = int rand (1+$i); # next if $index == $newloc; # Not needed in Perl. @$arrayref[$i, $newloc] = @$arrayref[$newloc, $i]; } }



The latest thing. Guess how it makes me feel.

Service orientation

Service orientation is a means for building distributed systems. At its most abstract, service orientation views everything-from the mainframe application to the printer to the shipping dock clerk to the overnight delivery company-as a service provider. Service providers expose capabilities through interfaces. Service-oriented architecture maps these capabilities and interfaces so they can be orchestrated into processes. The service model is "fractal": the newly formed process is a service itself, exposing a new, aggregated capability.

this link for unbuffered io for my tail program?
----------------------------
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/reading_input_buffer_events.asp ----------------------------



You have an administrators table (ID, firstname), a privs table (ID, name) and a modules table (ID, module).
The challenge is to return all privs for all modules for a given user. However, if the user does not have a privilege then a result must still be returned but the flag set to zer.
IE:
for user simon:

privname module flag test test 0 vest test 0 test bob 1 vest bob 0
So Simon has not privs on module test and one in bob. This is what I have so far
select apm.admin_ID, flag = case when apm.priv_ID is null then 0 else 1 end, apm.module_ID, p.name, m.module from ADMIN as a join APM_LINK as apm on a.ID = apm.admin_ID and a.firstname = 'simon' right join PRIVS as p on p.ID = apm.priv_ID left join MODULES as m on apm.module_ID = m.ID
This returns
adminID flag moduleID name module NULL 0 NULL create NULL NULL 0 NULL update NULL NULL 0 NULL delete NULL 1 1 1 super news 1 1 2 super directory
We can do away with adminID and moduleID at a later stage. Have fun!

The solution!! Taa daa!!
select flag = case when apm.admin_ID is null then 0 else 1 end, --m.ID as "module id", m.module, --p.ID as "priv id", p.name as "priv" from MODULES as m cross join PRIVS as p left join APM_LINK as apm on apm.module_ID = m.ID and apm.priv_ID = p.ID and apm.admin_ID = (select ID from ADMIN where firstname = + 'simon') order by m.module

Adso.pl
Recursive H
Free Nodelet freed
Perl and Address labels, oh yeah!
My Year in Perl, 2004 edition
Bringing Logic Programming to Perl
shared mem segments with Inline::C
Neural Nets
Re: Printing multiple arrays as multiple column
Higher Order Perl Mailing List
Convert PowerPoint Presentation to Word Document with Win32::OLE
Finding Windows XP CD Key
Visualize packed data as bits
inventory.pl
Tales from writing a RPN evaluator in Perl 5, Perl 6 and Haskell
Closed geometry: a train track problem
$picture eq ('word' x 1000)
Project Euler (a series of challenging mathematical/computer programming problems)
Re: Looking for help with AI::Genetic and classroom scheduling
A Beginning Guide To Evolutionary Algorithms
XY Problem
Operator Associativity and Eliminating Left-Recursion in Parse::RecDescent
:lvalue support in Object::InsideOut
Subroutine attributes for solving crosscutting problems
Evil Interview Questions
Log In?
Username:
Password:

What's my password?
Create A New User
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (13)
As of 2014-09-18 21:14 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    How do you remember the number of days in each month?











    Results (124 votes), past polls