Lori713 has asked for the wisdom of the Perl Monks concerning the following question:
I've been given the task of learning Perl, Javascript, HTML and SQL on my own in order to take over a program written by a consultant (currently about 8000 lines of code with little/no documentation) that generate financial reports on the Web (I also have to develop some new reports from scratch). The current program works and is in production. To read more about my experience:
I currently don't know any of these languages, but am reading various books and nodes here to get ideas. My previous programming experience involves a little Visual Basic (i.e., "Programming Lite"), and something called nVision from PeopleSoft (i.e., "Spoonfeeding Heavy"). I'm fairly comfortable with all the new languages except Perl. The next Perl class available that I can go to is in late October.
Did I mention a draft of these new reports are due October 31st?
I'm not sure how to ask my question because I know so little at this point, but I'll try...
How does one debug a Perl program containing multiple subroutines that generate HTML pages that use Javascript on the Web (and uses SQL to pull data from the database)? To read more about what I've done so far:
I've read numerous nodes and articles about the Perl debugger (i.e., nate's article on "using the Perl debugger," the idiot's guide to debugging node). I've also explored ActiveState's site and watched a couple of recorded webcasts about VisualPerl. I also have the Camel book, the Llama book, an HTML and Javascript book, a Perl/CGI book, all of which I've scanned for various tips and tricks. I've also tried digging through perldoc perldoc, perlfaq1 through -9 and other various manpages.
Confession: I don't know how to run my program from a command line when my access to it is through an URL I type in. I make changes to a text document, FTP it to the Unix server, and it automagically appears when I go into Windows Explorer and type in my URL (I'm on Windows 2000 on a client PC). I have found someone to show me how telnet works... but I'm not sure how to debug in that environment since my program/subroutines generate web pages...
Any thoughts on how to set up an effective debugging system given that I'm playing with multiple languages on the Web? I'm having trouble pulling together all the information I've read over the last couple of weeks.
Just FYI: The program I'm taking over doesn't use the "-w" or "use strict" etc. (when I put these helpful things into the program, well, it's ugly - lots of "uninitialized variables"). Any dangers to putting "my" on every variable? The consultant said a lot of the variables are global, but I'm not sure what that means in the context of my one subroutine library (everything is in one file). He named the global variables "g_xxx," so I do know how to find them. I'd like to be able to use the "-w" and "use strict" for future help in debugging.
Thanks for any help you can provide. If you need more info, please let me know. Lori
P.S. Sorry this is so freaking long but I wanted to provide all the info I'd seen people ask for in previous posts.
|
---|
Replies are listed 'Best First'. | |||||||
---|---|---|---|---|---|---|---|
Re: Sub-initiate needs help getting started
by bobn (Chaplain) on Aug 25, 2003 at 21:57 UTC | |||||||
Holy cow. Where to start? You have a hell of a job ahead of you. I'm not convinced it's doable. Had the consultant left a useable code base, it would be much easier - but with variables named g_xxx in an 8000 line file with no comments, it sounds like he intentionally obfuscated his code. His code will probably be almost worthless except as a negative example. Update: Use CPAN http://www.cpan.org - especiually the modules. You'll already be using CGI and DBI, but there may be many more there than can be uused to your advantage. As for the "from scratch reports", that's how I'd say you should do it - from scratch. Use the consuiltant's code only for understanding table layouts. Write yours from scratch, using strict and warnings, which can save you much aggravation. When in doubt, simplify. I'd stay away from javascript unless you know for certain that only one type and version of browser will ever be used. Even then, I'd stay away from it - it's best use is for validating user input w/o server involvement, but it's a whole other language to learn that won't necessarily get you closer to getting the job done. Validate in perl instead, because security requres that the input be validated at the server in any case.)Keep the HTML simple, too. Do not fall prey to the temptation to endlessly tweek the appearance of your web-pages. You don't have time, and it's futile anyhow - HTML is not meant for that, and it will look different in different browsers. Use the standards for HTML. Do not write browser specific code. That way lies madness. --Bob Niederman, http://bob-n.com All code given here is UNTESTED unless otherwise stated. | [reply] | ||||||
by agentv (Friar) on Aug 26, 2003 at 05:48 UTC | |||||||
...do not despair Lori713 in the face of this discouraging (and somewhat misguided) advice. It is not useful to conclude, "He drives without a seatbelt, therefore he must be a terrible and careless driver." And it is similarly unhelpful to conclude (without seeing any code) that the code is worthless and unmaintainable on the basis of a few superficial points of dogma. Don't get me wrong. I think that everyone learning or writing their own code from scratch should make a point of using strict and -w for everything they create. But my experience with production projects is that you should not feel stopped from proceeding if you cannot retrofit those civilities onto an existing code base. There are more important tests that you can conduct. But go ahead guys. Keep on declaring any code that does not meet your personal standards for cleanliness or elegance to be "worthless" and watch the number of IT managers who refuse to let Perl anywhere near their shop grow. (It's already a depressingly large number of them already.) The cliche' of a consultant who leaves behind a legacy of ugly and maintainable code is a pervasive one. And part of that legend is created by weak and dogmatic programmers who know how to write things from scratch, but quiver in the face of complex code written by someone else whose thought process works differently. And there's good reason for this. It's easier to pronounce code to be "worthless" than it is to admit, "I don't understand how this works." So don't be discouraged Lori. And as other advice here indicates, you should simply take small parts of the problem and examine them one at a time. As you work with the system, you'll begin to see what it does, and what you can do with it in the future. You've already identified several things that are important about your Perl parts of the problem. Use the seatbelt where you are creating new things by using -w and strict in your own code. Learn how to test your code from the command-line. In addition, begin now to learn how your web server operates. Become familiar with the configuration files (or other configuration systems) and with the log files that are created. Sometimes those log files are going to contain the clues about what went wrong with your system. Most of all relax. What you have to do can be done (in fact, you will do it) and you have already taken an important step by joining this community and becoming part of the conversation.
...All the world looks like -well- all the world,
when your hammer is Perl. | [reply] | ||||||
by bobn (Chaplain) on Aug 26, 2003 at 06:57 UTC | |||||||
But go ahead guys. Keep on declaring any code that does not meet your personal standards for cleanliness or elegance to be "worthless" and watch the number of IT managers who refuse to let Perl anywhere near their shop grow.
If the new reports are only trivially different from the existing reports, the OP might be able to tweak the program to meet the needs. Some parts of the program may be salvagable. But a system that is contained within one 8000 line file with few or no comments, doc, no warnings and no strict, and lots of global variables named in the form g_xxx is almost certain to be shit code. It is hard to imagine anyone who cared writing anything decent that matched this description. But it probably is a mistake to condemn it without seeing it. Lori, you might want to try and get some help with this. This is one place. Another might be at a meeting of your local Perl Mongers group. http://pm.org. The advice given above about learning how your webserver works is excellent, especially the log files, especially the error logs because anything you output to STDERR (eg warn 'something') ends up there instead of in the webpage. --Bob Niederman, http://bob-n.com All code given here is UNTESTED unless otherwise stated. | [reply] [d/l] | ||||||
by agentv (Friar) on Aug 28, 2003 at 05:18 UTC | |||||||
by dash2 (Hermit) on Aug 27, 2003 at 17:21 UTC | |||||||
First of all, the reason that managers are often turned off Perl is nothing to do with "dogmatic programmers". It is much more to do with incompetent programmers who don't use strict or -w, and who, partly for these reasons, do write unreadable code. It is possible that this consultant was a genius who doesn't need a seatbelt (see Paradigm Shift - Don't use strict for a famous example). But the 99% likelihood is that he was an idiot who didn't know how to use a seatbelt - especially if his code has a lot of globals, which is normally a sign of bad design. Finally, hard-to-understand code is almost always bad code. (There are exceptions, like tightly optimized loops, or JAPHs.) And for a consultant, this is doubly true: if you've left a lot of code that other people find difficult to use, then you're doing a crummy job - no matter how brilliant your code may be, your first priority was to help the people you worked for. Beginning programmers often think that if they don't understand something, it must be very clever, deep code. Usually the best code is very easy to read, almost like pseudocode. I think this is a tough job, but kudos to Lori713 for taking it on.
| [reply] | ||||||
by agentv (Friar) on Aug 28, 2003 at 05:34 UTC | |||||||
by dash2 (Hermit) on Aug 28, 2003 at 17:09 UTC | |||||||
Re: Sub-initiate needs help getting started
by Popcorn Dave (Abbot) on Aug 25, 2003 at 21:43 UTC | |||||||
Get yourself a copy of Perl in 21 days. That is a good starting point for where you're at. The books you do have are invaulable, but I personally think they can be a bit overwhelming if you've never done a lot of programming. Next, get yourself a copy of Perl - obviously. :) ActiveState's is good for windows. So it's sort of up to you whether you go with 5.6 or 5.8. Other monks here who have more experience with both versions would be better qualified to advise you on that. I'm using 5.6 and it works for me. Now as far as your debugger, there is a graphical debugger that is available on the web <a href="http://world.std.com/~aep/ptkdb/>here. Personally, I prefer this debugger, but again your mileage may vary. Also, as a beginner I would say yes, put in use strict, -w and perhaps even use diagnostics to help you catch errors. And remember there's a LOT of good people here that are willing to help. Good luck! There is no emoticon for what I'm feeling now. | [reply] | ||||||
by jdporter (Paladin) on Aug 26, 2003 at 04:02 UTC | |||||||
Get yourself a copy of Perl in 21 days.Ugh. Terrible advice. Of all the Perl books out there, this is one to avoid. For better advice on how to choose a Perl book: I personally would recommend the following excellent on-line Perl books: You can't learn Perl in 24 hours, 21 days, 12 weeks, 9 months, or a year. I've been programming Perl for nearly five years and I'm still learning.Same here. jdporter | [reply] | ||||||
by bart (Canon) on Aug 26, 2003 at 10:36 UTC | |||||||
There's also a title Debugging Perl. | [reply] | ||||||
by Popcorn Dave (Abbot) on Aug 26, 2003 at 04:14 UTC | |||||||
And yes there are a bunch of lousy Perl books out there, the Dummies Guide being one of the worst IMHO. However, I still think that for the person with little or no programming experience, O'Reilly tends to be a bit heavy. Just my opinion. There is no emoticon for what I'm feeling now. | [reply] | ||||||
by jdporter (Paladin) on Aug 26, 2003 at 16:11 UTC | |||||||
by bobn (Chaplain) on Aug 25, 2003 at 22:03 UTC | |||||||
Next, get yourself a copy of Perl - obviously. :) ActiveState's is good for windows. The OP stated that the app is already up and running on a Unix box. AS might be useful to take home on a laptop for practice, though. --Bob Niederman, http://bob-n.comAll code given here is UNTESTED unless otherwise stated. | [reply] | ||||||
by Popcorn Dave (Abbot) on Aug 25, 2003 at 23:38 UTC | |||||||
There is no emoticon for what I'm feeling now. | [reply] | ||||||
by bobn (Chaplain) on Aug 26, 2003 at 05:26 UTC | |||||||
by Lori713 (Pilgrim) on Aug 25, 2003 at 21:52 UTC | |||||||
| [reply] | ||||||
Re: Sub-initiate needs help getting started
by dbwiz (Curate) on Aug 25, 2003 at 21:52 UTC | |||||||
On learning Perl, woolfy has recently written a node on the available sources (Where and how to start learning Perl). Also, have a look at our Tutorials. About SQL, if you want a start, here is an online tutorial: SQL for web nerds. For HTML, there is the barebones. About javascript, I don't know, sorry, but I think you have enough to read so far ... :). | [reply] | ||||||
Re: Sub-initiate needs help getting started
by CombatSquirrel (Hermit) on Aug 25, 2003 at 23:07 UTC | |||||||
The partial documenting "on the go" is a bit more work, but it'll pay offonce you wonder what arguments should be passed to that what-was-it-called function. And the last thing: Whenever you have problems with a piece of code that you think should work, but doesn't, post it and you'll see (if it is a sensible small problem - splitting ;-) that we try to help with code as much as with theory. Hope this helped. CombatSquirrel. Update: About the HTML part: Although you probably don't, if you should know German, have a look at Stefan Münz's Selfhtml tutorial; it is simply the best HTML tutorial I know. Entropy is the tendency of everything going to hell. | [reply] | ||||||
Re: Sub-initiate needs help getting started
by liz (Monsignor) on Aug 25, 2003 at 21:49 UTC | |||||||
And best of luck. But with the help of the Monastery, it should work out ok. I wonder how much more expensive the not-documenting, using global variables consultants were. Liz Update: Removed "making" in the last line. | [reply] | ||||||
Re: Sub-initiate needs help getting started
by williamp (Pilgrim) on Aug 25, 2003 at 22:48 UTC | |||||||
| [reply] | ||||||
by shenme (Priest) on Aug 25, 2003 at 23:33 UTC | |||||||
What does .*? mean? Try it with and without the '?' (pardon the Windows quoting): And even when you don't understand it, at least you'll know what it did ... | [reply] [d/l] | ||||||
Re: Sub-initiate needs help getting started
by skyknight (Hermit) on Aug 26, 2003 at 00:03 UTC | |||||||
The short answer is, and I'm not trying to be an asshole, that you are thoroughly screwed. There is nothing worse than being handed code that was shoddily written by an untalented hack of a consultant. It is a huge mistake for companies to hire random people without any regard for their talent level and then dump the maintenance onto some unwitting internal developer. The code that the consultant hands over may mostly work for a very specific task, but it will probably be extremely difficult to extend, and god forbid you should ever have to debug 8000 lines worth of spaghetti. Many of the skills that are essential to being a good software engineer, and that go largely uncovered in school, involve social interactions. More than anything else, you have to be honest, and you have to rein in the unreasonable, illogical expectations of managers. Do not let managers make unreasonable promises, and do not make grandiose claims about how you'll learn four new languages, debug 8000 lines of code, and cure both AIDS and World Hunger in two months. When you fail, you'll look foolish, your manager will look foolish, and the clients will be hung out to dry. Sometimes when you are handed a true monstrosity, it is better to glean what info you can from it, and then burn it to the ground and start over again. Given your gruesome description of the task at hand, I think that you should seriously consider this course of action. | [reply] | ||||||
by chantstophacking (Acolyte) on Aug 26, 2003 at 05:57 UTC | |||||||
The epitome of effectiveness is to accomplish a goal without trying. | [reply] | ||||||
Re: Sub-initiate needs help getting started
by Cody Pendant (Prior) on Aug 26, 2003 at 04:10 UTC | |||||||
I'm going to join the chorus of "good luck" and also "the consultant should be shot". I'm also going to say you're in a very sticky situation. How's this for a question -- why are you attacking this problem from behind? You've got 8,000 lines of bad code -- forget that for a second, what is it that you want to achieve? If you've got a database and you've got some expected output, we might well be able to bridge that gap with a new script. Is the JavaScript necessary, or does your boss just need to see reports on x number of widgets sold on the east coast versus x number of widgets sold on the west coast? Considering what we know about your consultant, they might very well not be very good reports in the first place. So I'm saying, what if you go back to your boss and say "forget the exact details, layout etc of the reports you have now, reproducing them is a very complex job -- what do you really need to know? I'll start on that." and take it from there. Getting data out of a database into a web page is not the hardest task you will ever face in Perl.
| [reply] [d/l] | ||||||
by blackstarr (Friar) on Aug 26, 2003 at 07:20 UTC | |||||||
Forget the look & feel until you are answering the correct questions ... then the look and feel will be easy.
Good Luck & ... | [reply] | ||||||
by markguy (Scribe) on Aug 26, 2003 at 18:10 UTC | |||||||
In any case, I don't think it's an answerable question short of seeing the code itself and ideally the functional specs for what the code is apparently solving. Figuring out legacy code can do wonders for jumpstarting your knowledge base (it's how I got started) and limits the feeling of helplessness that can hammer you if you have no experience in handling this sort of situation. This may not fit the goals of the original poster, but you might consider hiring another consultant. And checking if they have a handle here for good measure ;). Hell, even if the company won't go for it, it might be worth paying some amount of your own personal money to get a little hand-holding as you work through it. Or just abuse the hell out of SoPW for the cheap, yet amazingly effective method! | [reply] | ||||||
Re: Sub-initiate needs help getting started
by bobn (Chaplain) on Aug 26, 2003 at 03:28 UTC | |||||||
Any dangers to putting "my" on every variable? It's not always going to work. Here's a (contrived) example: Results are: $x is 3Now look at: Results are: $x is Reason: the my makes $x a "lexical" varible which is "in scope" only within the innermost { } pair. Ouside of there, it has no meaning, or other, unrelated meaning. So you'll at least have to dig out all the globals and do the my at the filde scope level (eg outside of any { } blocks). They'll still be globals, but it lets you start using strict. --Bob Niederman, http://bob-n.comAll code given here is UNTESTED unless otherwise stated. | [reply] [d/l] [select] | ||||||
Re: Sub-initiate needs help getting started
by CountZero (Bishop) on Aug 26, 2003 at 05:56 UTC | |||||||
A good way to start (other than regularly visiting the Monastery) is to dig out the original/amended/updated specs of this project, so you can find out what it was meant to do and (perhaps) how it should look. This may help you to understand the program (if it was written to conform to the specs, that is) and more easily find your way into it. It may perhaps even show you that it is better to start over from scratch. This may seem a daunty task, but with the help of some templating systems (Template-Toolkit or Mason or another templating system (super-search the monastery to find more) it may even be less demanding than finding your way into the undocumented sub-standard code of someone else and at least it let you adapt future versions of this script with less effort. Oh yes, and if you found the original specs, see if you can find how much the consultant charged for his "work". Perhaps you can negotiate a bonus along the same lines once the job is finished ;-) and don't forget to drop a small bit of your bonus in our Offering Plate. CountZero "If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law | [reply] | ||||||
Re: Sub-initiate needs help getting started
by PodMaster (Abbot) on Aug 26, 2003 at 11:33 UTC | |||||||
How much time can/are you going to devote to this? I say give it a week, and then think about getting some help (a week ought to be enough time to help you decide if you can meet the deadline on your own).
| [reply] | ||||||
Re: Sub-initiate needs help getting started
by Lori713 (Pilgrim) on Aug 26, 2003 at 13:03 UTC | |||||||
THANKS AGAIN! | [reply] | ||||||
Re: Sub-initiate needs help getting started
by bm (Hermit) on Aug 26, 2003 at 13:44 UTC | |||||||
Best of Luck! -- bm | [reply] | ||||||
Re: Sub-initiate needs help getting started
by wirrwarr (Monk) on Aug 26, 2003 at 08:04 UTC | |||||||
I assume the unix box uses Apache as a web server. So go to apache.org, download the relevant version (either 1.x or 2.x, depending on what's installed on the unix box), and install it on your windows box. If you have questions about this, a search on the web ("install apache and perl on windows") should give you plenty of help. Don't panic! | [reply] | ||||||
Re: Sub-initiate needs help getting started
by jonadab (Parson) on Aug 26, 2003 at 14:02 UTC | |||||||
Any dangers to putting "my" on every variable? Yes, don't do that. When you write your own code from scratch, you can write it to use all lexical variables, but if you try to retrofit that onto code that wasn't written that way you can get strange results. The "always use strict" advice only applies to new work; you shouldn't try to retrofit it to someone else's code. At least, not until you're confident you know what you're doing. Warnings you should use. Now, for cgi you don't want warnings on for production use, because they'll mess up the interaction between the script, the web server, and the browser. So you can either test from the command line, or use one of the modules that sends errors and warnings to the browser. Or you can turn on warnings while you're debugging and turn them off again after, which is probably what I'd do. You do know how to tail web server logs, right? My long-term advice to you is to plan on gradually rewriting the whole thing (or most of it) yourself, in bits and pieces. You'll have an easier time maintaining it that way. But you don't have time to do that all at once right now, so only write the bits you need to write in order to meet your deadline. Later you'll want to add or change more things, and you can (re)write more parts of it at that time.
| [reply] [d/l] | ||||||
Re: Sub-initiate needs help getting started
by Wally Hartshorn (Hermit) on Aug 26, 2003 at 14:59 UTC | |||||||
How does one debug a Perl program containing multiple subroutines that generate HTML pages that use Javascript on the Web (and uses SQL to pull data from the database)? If you're very lucky, he used some sort of HTML templating module. (HTML::Template is probably the easiest to get started with.) That can help immensely because it avoids having oodles of print statements spewing out HTML code at numerous places in your code. Unfortunately, I suspect you aren't that lucky. If you're lucky, the 8,000 line file is done as a separate module (something.pm), with just a short perl program calling the initial subroutine in that module. If that's the case, it's easy to write short perl programs of your own to call subroutines in that module to test things out. For example, if there is a subroutine in the module that accepts an employee name as a parameter and returns that employee's salary, you can easily write a short perl program to call that subroutine, pass it a hard-coded name for testing purposes, and then print out the salary that is returned on the command line. That allows you to debug just that tiny segment of code from the command line, rather than dealing with the entire CGI environment in a web browser. Unfortunately, I suspect you aren't that lucky in this bit, either. If the changes you will be making are quite minor, I would recommend making the changes first and verifying that they work before doing any significant structural work. If the changes are going to be extensive, then I would suggest restructuring the existing code first to make it easier to make the future changes. Suggested modules: CGI.pm, DBI.pm, HTML::Template, Config::IniFiles, Test::More, HTML::FillInForm, CGI::Application, CGI::Application::ValidateRM. These let you easily read data from a form (via CGI.pm), read from the database (via DBI.pm), display values in an HTML page (via HTML::Template), read from a configuration file (Config::IniFiles), run automated tests to ensure that a routine that you had working earlier hasn't been broken by later changes (Test::More), pre-populate a web form (HTML::FillInForm), handle CGI interactions via run-modes (CGI::Application), and validate data entered in a form (CGI::Application::ValidateRM). An 8,000 line program without "use warnings" and "use strict" that incorporates HTML, JavaScript, and SQL without modules and automated tests is pretty much a textbook example of what NOT to drop on someone who is new to Perl. Look on the bright side -- every Perl project you have after this one will be easier! :-) Wally Hartshorn | [reply] [d/l] | ||||||
Re: Sub-initiate needs help getting started
by dragonchild (Archbishop) on Aug 26, 2003 at 15:33 UTC | |||||||
Note: All of these suggestions are based on the assumption that this code will continue to be maintained and extended for a long time. If it's going to be retired soon, ignore everything I've said, do what you need to do (no matter how dirty), and be glad the monstrosity is dying a well-deserved death. Good luck. If you ever need to vent/ask/whatever, please don't hesitate to come here. :-) ------ The idea is a little like C++ templates, except not quite so brain-meltingly complicated. -- TheDamian, Exegesis 6 Please remember that I'm crufty and crochety. All opinions are purely mine and all code is untested, unless otherwise specified. | [reply] | ||||||
by MidLifeXis (Monsignor) on Aug 27, 2003 at 18:14 UTC | |||||||
3. That test-suite thingy - make one. DO NOT DO ANY WORK ON THE CODE WITHOUT ONE. Any recommendations or literature on doing this? I have a sysadmin / jack of all trades type background, and as such, am starting to reach the limit of my "how to test" knowlege. I am looking for some insomnia meds in the form of some heavy (or light) reading on this topic. Thanks | [reply] | ||||||
by dragonchild (Archbishop) on Aug 27, 2003 at 19:31 UTC | |||||||
A lot of these suggestions assume that you have a rational codebase. For example, if you cannot extract a given subroutine from its context and test it, you very well could have a problem. Also, your testing design ------ The idea is a little like C++ templates, except not quite so brain-meltingly complicated. -- TheDamian, Exegesis 6 Please remember that I'm crufty and crochety. All opinions are purely mine and all code is untested, unless otherwise specified. | [reply] | ||||||
Re: Sub-initiate needs help getting started
by talexb (Chancellor) on Aug 26, 2003 at 15:17 UTC | |||||||
Having written some truly ugly Perl back when I was starting as a consultant, I understand how a little 250 line script can grow to 8000 lines when the person who's paying the bills says things like "Re-write? Are you crazy? We don't have time! Stick in this new feature, and have it done by Tuesday!!". You've already received some great advice -- just pop back here when you run out of brain cells and we'll help you as best we can. Probably the best advice is to version control as much of this as you can -- being able to 'roll back' to the previously working version is priceless. If you're running Linux, check out rcs -- very simple, but very solid. Failing that (like, if you're on Windows), make new sub-directories for each version. Not quite as convenient, but still very useful. Good luck! --t. alexLife is short: get busy! | [reply] [d/l] | ||||||
by Rudif (Hermit) on Aug 26, 2003 at 20:48 UTC | |||||||
[reply] | |||||||
Re: Sub-initiate needs help getting started
by johndageek (Hermit) on Aug 26, 2003 at 20:44 UTC | |||||||
Base unsderstanding - the evil scary program runs and does it's thing in an acceptable manner. You have little or no programming experience, let alone how these systems fit together. There is a systems person or group who understand and keep the database and web servers al running and in tip top shape:) If the above is not true and you are the sole IT person, or most knowlageable it person the company has, let us know and we will try a different tack. You do not have time to flail around learning perljavascriptsqlhtmlandotherbuzzwords.
First priority - get the following information: Now small step 1) The database, this is your friend in whom all good things are stored. You will want a client connection to the database (sqlplus, toad or torra for oracle). You will use this tool to access the database, get information about how tables are set up, and test your sql statements. Have them set you up with an account for the live database and the test database - you do have a test database I hope??? Ask the database people how to list the tables and columns in the database you will be using. Once you can see the tables and column names, try a select clause to pull some of the data out. You will need to find how the tables are connected. Run the evil report then try to pull the same information with sql. Having pulled the data successfully (yes I would suggest several tests) Write some perl code using DBI to connect to your database using the same sql statements from above. Dont worry about html or formatting, just get the perl program to pull the data and print it out from a command line. Success! Now all you need to do is format with html, our esteemed local monks are much more familiar with packages to help you in this area than I am.
Good Luck! | [reply] | ||||||
Re: Sub-initiate needs help getting started
by qq (Hermit) on Aug 26, 2003 at 22:36 UTC | |||||||
Good luck, first of all. There is a lot of good advice given so far about how to write better code than you've been given, but I'd be wary of making large structural changes given your deadline. It certainly needs it, but I doubt its feasible in the time frame. So here is a quick and dirty approach: Do the subs all look roughly the same - if you squint? It may be that there is just one main sub per report and some helper functions. Given that the code is so long, its probably not well structured and may be very repetitous. If the new reports you need to create are similar to the existing ones, you can just copy and paste one of the existings subs and make modifications until it works. If the subs don't all look the same, and aren't 'one per report', this approach won't work at all. Another possible route is to not change the existing code at all, but write any new report code from scratch. In the .pl file have a simple exit clause near the top so that if one of the new reports need to be processed, you use the new code, or else fall through to the original mess. If the existing code is considered broken, this won't work. Two more random points: And again, good luck. | [reply] | ||||||
Re: Sub-initiate needs help getting started
by Lori713 (Pilgrim) on Aug 26, 2003 at 16:00 UTC | |||||||
I wasn't very clear in describing the current "program" I'm taking over. There are two main files, one is the .pl that starts the processing, and the other is a .lib that contains all the subroutines. The .lib file is the monster with about 8000 lines of code. There are not a lot of comments in the actual .lib file (I will be adding these as I figure out what each subroutine does). Without getting into the politics, my manager had no say over what this programmer did, or left behind. We just ended up being the ones to maintain it. The global variables do begin with "g_" and the "xxx" part was me being lazy... There are some decent variable names in there (but I'm often confusing them with the "m_" variables with the same "xxx" part). Another confession: I don't know how to build a test suite, but I'll dig around this website for ideas before asking for input right now on this node. The subroutines are mostly Print statements to get the HTML to generate on the Web page; I don't see where any HTML template module was used. I have figured out how to do a HERE on my own (I think that's what it's called) so I won't have to type PRINT all the time. I have also looked a little at some of the HTML modules, and I think I'll try to use them. A lot of the new reports are very similar in look and feel to the existing ones. At this point (and the looming deadline), I'll probably copy one of the reports, clean it up, and use it for a template for the new reports. I'm hoping that by putting my new reports in a separate file I can use -w and use strict. Thanks for even more great pointers. I'm actually getting a little jazzed about starting in on this wooly mammoth! | [reply] | ||||||
by dragonchild (Archbishop) on Aug 26, 2003 at 16:14 UTC | |||||||
As always, feel free to post more questions, ideas, concerns, etc. Many of us (myself included) are more than willing to help you 1-on-1. Feel free to ask! :-) ------ The idea is a little like C++ templates, except not quite so brain-meltingly complicated. -- TheDamian, Exegesis 6 Please remember that I'm crufty and crochety. All opinions are purely mine and all code is untested, unless otherwise specified. | [reply] [d/l] | ||||||
by duffbeer703 (Novice) on Aug 26, 2003 at 17:50 UTC | |||||||
| [reply] | ||||||
by CountZero (Bishop) on Aug 26, 2003 at 20:13 UTC | |||||||
On the issue of IDE's and if your company can spare a few hundreds of dollars, the professional version of Komodo certainly warrants a closer look. I'm quite happy with it as it really speeded-up my programming. Of course, YMMV and many other Monks will surely tell you that the IDE they are using is far superior (and less expensive), but I'm happy with it and that's all I can say. CountZero "If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law | [reply] | ||||||
by dragonchild (Archbishop) on Aug 26, 2003 at 20:20 UTC | |||||||
by CountZero (Bishop) on Aug 26, 2003 at 20:36 UTC | |||||||
You may be not totally lost yet! The "small" .pl program probably works as some kind of scheduler, calling the appropriate sub-routines from the .lib file as and when needed. Mind you, using a .lib file instead of a properly packaged module (.pm) is certainly not considered a standard in the Perl-world. Anyhow, the fact that subroutines are used in this .lib file is something "good". Indeed, the way to go forward is trying to figure out what each sub-routine does. Even more important is to find out how the flow of execution runs through all these subroutines. Knowing that, you can establish some hierarchy, see what data is passed around (in and out of the subroutines; do not forget that "global" variables may muddle/mess up this picture!) and what gets send to the web by which subroutine. Having discovered all this you can then proceed, to carefully take apart the things which need to be changed and apply the necessary changes. It may seem like a chore but all this preparative work will help you later. 'Time spent in reconaissance is seldom wasted' they told us in the Army (to which one replied "A good scout is a dead scout" (Gen. Sherman, if I'm not mistaken)) CountZero "If you have four groups working on a compiler, you'll get a 4-pass compiler." - Conway's Law | [reply] | ||||||
Re: Sub-initiate needs help getting started
by bobn (Chaplain) on Aug 26, 2003 at 15:05 UTC | |||||||
After re-reading the original post, I think I may have inferred more than I should have. Specifcally, I still think you have a very steep learning curve to climb, in that you have to learn a tremendous amount in a very short timeframe to be able to either modify the existing code with good understanding, or to write original code, however simplified, to make the new reports. In addtion to perl, SQL, HTML and (possibly) javascript, you'll also need to understand webserver function, and a litle Unix experience at least in order to debug, so I am still not convinced that either of these is possible, on your own, by October 31st, even devoting full time to it. --Bob Niederman, http://bob-n.comAll code given here is UNTESTED unless otherwise stated. | [reply] | ||||||
Re: Sub-initiate needs help getting started
by barbie (Deacon) on Aug 27, 2003 at 10:51 UTC | |||||||
WAMP (Windows-Apache-MySQL-Perl) is just as capable as LAMP (Linux-Apache-MySQL-Perl) for development. Okay there are some Linux dependant features that you can't use, but in the main most of what you are likely to want is portable between Windows and Linux. Apache (1), MySQL (2) and Perl (3) are all available for the Windows platform and with a little effort can all work together.
(1) http://www.apache.org/ You didn't mention which database you were using, so if not MySQL you might not be as successful at finding a Windows version. Although if your existing database is accessible from your desktop, then you won't need to have a database on your desktop, unless you want it for the learning experience. DBI can connect to a database anywhere, as long as it has permission. If you don't currently have access to your database, it may only require a quick update to the permissions table in the database to let you in. Ask the company's Database Administrator if in doubt. In the short term there is likely to be a steep learning curve in putting all this together, but in the long run you will have a good experience of figuring out how Apache config files work and getting Perl talking to a database. Another poster mentioned having a Version Control System, which is a must just in case anything goes wrong. CVS (4) is available on Windows, which is good for basic projects.
(4) http://www.wincvs.org/ Once you've set everything up, the turn around for developing and testing can be much quicker. At my last company, I managed to get the designers, graphics bods and developers all using the same CVS repository and database, but with their own copies of the websites and web servers. They were then able to run the scripts and see the results for themselves before commiting to CVS and then testing on the test server. The subsequent projects were quite often finished upto 50% quicker. Debugging a CGI script can be painful. However, if you are using Apache then the error logs can be a godsend. However, a quicker error report can be obtained via the use of: use CGI::Carp qw(fatalsToBrowser);Place the above in your CGI script, and any fatal errors will get sent straight to the browser. Though ensure it's remove or commented out for production code. If you want to use your own trace logs, without interfering with the web server log files, have a look at Log::LogLite. A straightforward mechanism for recording processing information, with the added ability to decide the priority of a message, thus enabling you to filter what message to trace without having to rewrite huge chunks of your code. Alternatively, is it possible run portions of the script stand alone? If so then test scripts, like the ones found in many CPAN distributions may be a guide to help you. Test::MockObject is a great help if you want to test code without having a connected CGI or DBI interface. Several posts have mentioned templating. If all the HTML appears in your code, whether via straight prints or using 'Here Documents', then this can create all sorts of problems. It maybe too overwhelming to transpose these into a templating system now, but it is definitely something to think about. There are several good templating modules/systems recommended here and it would be a good idea to have a look through them to see what suits you best. By having a templating system you may have found that much of the code to create each report is actually duplication of effort. Thus a new report can be as simple as writing a new subroutine that extracts the data and stores it in a format ready for the presentation process, then creating a new template to present the data. Quite possibly a half a day job or less for each report. And what happens when your boss decides it has to be in XML format or any other format in the future? Templating allows you to change the presentation layout without any changes to the data extraction and preparation. Learning HTML and JavaScript isn't too demanding and there are many books and online resources which you may have found already. However, I would concur with others that JavaScript is not something you should be too eager to rely on. There are many aspects of JavaScript that are useful, such as the form validation aspects, but anything that comes from the outside world should still be validated server side too. 'Taint' is the keyword here. Node 80037 might be of interest. From your subsequent post I suspect you're on the right track already and may have found your job of producing the new report a little easier than you first thought. There is much food for thought here, so I hope it doesn't overwhelm you. Good luck and welcome to the community. -- | [reply] [d/l] | ||||||
Re: Sub-initiate needs help getting started
by Lori713 (Pilgrim) on Aug 27, 2003 at 13:40 UTC | |||||||
(1) I've loaded Eclipse and the Perl editor plug-in, and I've gotten it to work. It's given me a lovely list of modules and subroutines. (2) We are on Sybase 12, Apache for the Unix box web server (I think - see (6) below), the reports are generated on the internet, and the reports present financial info (i.e., balance sheet, income statement). (3) I've ordered some books that should be here this week (Debugging Pearl, the cheetah book) using my own funds. I work for the State, and we have NO money right now (they've actually starting laying off folks recently). (4) The DBI module is not currently being used but I think there's a connection established using the Sybase::CTLib module. (5) I am not the IT/DBA; I'm a reporting specialist who develops financial reports for our campus community. (6) If I understood correctly, I can set up my desktop to run scripts. I currently FTP the edited text file to the Unix server, and then open a browser and type in my URL. This is not a difficult process, but it can be annoying at times. Confession: I'm not sure how all this stuff works together (i.e., the web server versus the Unix server versus the database server (BUT I will look this up on my own later today). I also plan to download Apache to my PC once I confirm that our Unix box uses that for its web server. (7) There is minimal javascript being used, and it's pretty straightforward and understandable. I still have concerns about security (about which I know NOTHING at this point), but I'll talk to our DBAs about security issues later this week to make sure I'm covered. (8) I have learned (painfully through experience) how vital version control is, so... 'nuff said on that.
Questions: (a) What is the difference among the following: module, function, subroutine, script? I'd like to make sure I'm using the right terms (and understanding some of the advice given). CombatSquirrel mentioned creating some modules to help organize things. Is it best practice to have multiple "files", or have this similar-report generator all in one file? I'd appreciate advice on this. The main menu will have 7 reports, with a spot for input and a drop-down to select an accounting period. The reports will pull similar data and have a similar look, but there are differences among them. (b) Is there any way to pull a list of variables used within a file/script/program? The Eclipse editor does a nice job of finding the subroutines and modules, but I haven't found a way to pull variables (I'd like to understand where these variables are initialized, how used, etc.).
Whew! Enough for now. I'm going to have to pitch a tent to block the evil fluorescent lights...
Thanks for all for the input and advice! Lori | [reply] | ||||||
by dragonchild (Archbishop) on Aug 27, 2003 at 14:17 UTC | |||||||
As for your design, I would reccomend the following: ------ The idea is a little like C++ templates, except not quite so brain-meltingly complicated. -- TheDamian, Exegesis 6 Please remember that I'm crufty and crochety. All opinions are purely mine and all code is untested, unless otherwise specified. | [reply] | ||||||
by CombatSquirrel (Hermit) on Aug 27, 2003 at 14:36 UTC | |||||||
To (b): I think there is a way, but I could not get it working :/. Hope the others have ideas... Hope this helped anyways. CombatSquirrel. Entropy is the tendency of everything going to hell. | [reply] [d/l] | ||||||
Re: Sub-initiate needs help getting started
by pboin (Deacon) on Aug 27, 2003 at 13:49 UTC | |||||||
One side that IMO hasn't been addressed much, is your responsibility to yourself and to your management to get their expectations exactly where they need to be. If I were in your shoes, I would have expressed *huge* doubts about the October deadline. Maybe you'll make it, and maybe you won't, but you should be telling them when it can be done, not have them tell you when it *will* be done. You may piss some people off. (I've literally had people red-faced and jumping up and down.) However, once you do it a few times, and people know your word is good, life becomes much easier on you, and your organization will be better for it. Do not bow to pressure -- the truth will set you free. | [reply] | ||||||
by Lori713 (Pilgrim) on Aug 27, 2003 at 14:08 UTC | |||||||
On the other hand, the best you can do is the best you can do, and I try to duck under the political radar. | [reply] | ||||||
Re: Sub-initiate needs help getting started
by PERLscienceman (Curate) on Sep 03, 2003 at 23:23 UTC | |||||||
www.perl-books.com I thought that I would bring these to the forefront so that you could have easy access to them. I am also partial to the O'Reilly "Animal Books" I have found everyone of their books that I have encountered to be quite useful so here is their main link: Also... Take a look at Book Reviews and see what Perl Monks have to say about Perl related books. Anyway good luck... I am certain that you will find your fellow Perl Monks to be a great resource! I know I have! | [reply] |