Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

How to calculate development time?

by Siddartha (Curate)
on May 31, 2001 at 15:03 UTC ( [id://84516]=perlquestion: print w/replies, xml ) Need Help??

Siddartha has asked for the wisdom of the Perl Monks concerning the following question:

I'm not sure if this is the right place to ask my question, but here goes:

I have been asked to develop a website for a company. It will basically be a jobsite where users can register, upload CV's etc. and Employers can post jobs that are searchable by different criteria.

I don't have the detailed spec yet, but need to know how to estimate a delivery time.

This project will be done after hours, since I am working full time. I need to figure out how much time/day I can commit and how long it would take to complete.

I have decided that the cheapest and best option would be to use Perl and MySQL, hosted by an ISP.

Because it is hosted by an ISP, I don't have control regarding the installed Perl modules. It would have liked to use Mason or something similar using mod_perl but I'm not sure if it is, or will be installed.

I am not a Perl guru at all. I have only been using it up to now for basic CGI scripting. I have used MySQL before, but it is going to take me a while to define the database structure and the interfacing.

Any comments on calculating development time, or possible ways of implemeting something like this would be really appreciated.

Thanks
-Siddartha

Replies are listed 'Best First'.
How to unintentionally lie to your client and look like a chump when it's all over
by petdance (Parson) on May 31, 2001 at 17:48 UTC
    You're setting yourself up for spectacular failure. I mean full-blown, 4th-of-July-level spectacular.

    I don't have the detailed spec yet, but need to know how to estimate a delivery time.

    You can't. Any number you give the powers that be will be bogus.

    Them: "I need you to drive me somewhere. How long will it take?"
    You: "Well, where are we going?"
    Them: "Oh, just out of state."
    You: (thinking... "Hmm, not detailed, but I have an idea.") "Oh, maybe a day."

    ...later...

    Them: "OK, we're driving from New York to San Diego. And we'll be there tomorrow, right? 'cause if we're not, it's your ass."
    You: "Uh-oh."

    I have decided that the cheapest and best option would be to use Perl and MySQL, hosted by an ISP.

    How could you have made that determination without a detailed spec?

    You: "We'll be making this trip in my Yugo."
    Them: "Oh, by the way, I have a few boxes of stuff I have to bring. Well, OK, a lot of boxes of stuff. And a refrigerator."
    You: "Uh oh."
    Them: "And we'll be there tomorrow, right?"

    The end result?

    Them, to others: "Man, all I wanted was a little trip, and he just screwed the whole thing up, and it took forever to get there. I wouldn't use him again if you paid me."

    Run, do not walk, to the bookstore (or Fatbrain) and buy yourself a copy of Steve McConnell's excellent Software Project Survival Guide. Read it. Then, reconsider the wisdom of estimating anything without knowing exactly what you're estimating.

    xoxo,
    Andy

    %_=split/;/,".;;n;u;e;ot;t;her;c; ".   #   Andy Lester
    'Perl ;@; a;a;j;m;er;y;t;p;n;d;s;o;'.  #   http://petdance.com
    "hack";print map delete$_{$_},split//,q<   andy@petdance.com   >
    
      >I don't have the detailed spec yet, but need to know how to estimate a delivery time.

      You can't. Any number you give the powers that be will be bogus.

      Man I wish I could ++ you more than once for that statement. I've seen that too many times.

      Death March by Edward Yourdon is also a good book to read regarding Development projects.

      Aiiii!

      Closely related to the Scotty principle; tell them 3 times longer than it's going to take, then finish it early and say "yeah, things slipped into place a little sooner than expected". Trick to making this work: ensure that you end up charging less, too.

        The problem with the Scotty method is that your project might well take 6 times longer than you think it's going to take.

        Also, Scotty had reasonable requirements. "Give me warp speed in 3 minutes or we're all dead" is a clear requirement.

        xoxo,
        Andy

        %_=split/;/,".;;n;u;e;ot;t;her;c; ".   #   Andy Lester
        'Perl ;@; a;a;j;m;er;y;t;p;n;d;s;o;'.  #   http://petdance.com
        "hack";print map delete$_{$_},split//,q<   andy@petdance.com   >
        
Re: How to calculate development time?
by Brovnik (Hermit) on May 31, 2001 at 15:24 UTC
    I won't try to answer all of the issues here, because it would take all day.
    Your first problem is here.

    I don't have the detailed spec yet, but need to know how to estimate a delivery time.

    If you don't have a detailed spec, it will be difficult to estimate the delivery time.

    Start by deciding how much work needs to go into producing the detailed spec.

    You may want to agree that development as a set of milestones, e.g. :

    1. Rough Specification
    2. Detailed Specification
    3. Initial Design
    4. Detailed Design
    5. Initial Development
    6. Revise specifications
    7. Final Development
    You may only be able to give rough estimates for more than the first few, but you should be able to refine the estimates as you get further into the project.

    Clearly you need to know the capabilities of the hosting platform so that you know what tools you will be able to use.
    If you have to build some functionality from scrath, it will obviously take longer than if you can uise existing tools.

    One way to get ideas is to find a list of existing sites and get the client to give you feedback as to whether they like some of the features, or how they would like it to be different. That can often give you a more concrete feel for the detailed requirements.

    These are just a few thoughts. You really need to work up a detailed list of questions to make sure that when you start, your expectations about what you are going to deliver are the same as the client's.
    --
    Brovnik

      From Murphy's laws for programming:
      If something is easy, it'll take only twice as much as you thought.
      If it is difficult, it'll take at least ten times as much. : )

      No jokes now, project time estimate is a serious matter.

      I agree with the steps above, and I offer useful 'trick':
      Make customer aware that his/her thinking time also needs to be included, too.
      I.e. "I'll need 2 weeks to develop Detailed Specifications *after you finished* reviewing and we agreed on Rough Specs."

      Of course you will try to start 'Detailed Specs' right after finishing 'Rough', without waiting, but you need to won time… Like in chess game, to won you need to use your opponent’s time to think about your own next step.

      Obviously if you thought you need about 2 weeks to do something and they wasted 10 days reading specifications and responding to your questions, you still need 2 weeks to finish the job, right?

      Also, take into account you need some time in the evening to ‘recall’ where you finished last time. I call it ‘time for swap’. And like with disk swap, the more often you swap, the less time is available for a useful work. Looks like a lot of weekend work to me.

      You may also ask questions:

    • How complex is the project? Does it involve new skils? Do you want to learn these skills for free, or you do not care?
    • Is it for the same employer you are working daytime? Same manager, same computer, or other? It might be bad.
    • Do you have good relations with 'night project' customer? Afraid to stain them?
    • Who will be available to answer to night project questions, and when?
    • Who will decide quality of project completion? Penalty for missed deadline
    • Is anybody else involved in the implementation? If yes, how much control do you have over other developers?

      I read a horror story about case when guy agreed to program in night something for his daytime company, was asked to take another guy to participate, then the other guy got behind schedule. To make for missing time he started sneaking into ‘night project’ during the day, and his day manager was not excited, because he wanted him to work overtime for day project, what was also behind schedule. As it happens, specifications were changed a little, he missed deadline a little, was required by his night manager to accept penalty…

      As a result, he left company with bad feelings with his day time colleagues and manager, and was not paid fair price.

      His advice: don’t do that, if you do not have full control.

      My advice: You’ve been warned. : )

      pmas

      I think this order of tasks is pretty good (especially because it "plans to throw one away") but incomplete. People (even technical managers) often completely omit testing time from project estimates. It is as if once the code is written the customer can deploy that afternoon.

      I can see some naive justifications for this, but I have experience to the contrary:

      Testing is done during development so it does not need to be scheduled.

      Developers rarely test their own code this thoroughly, especially when they are on a deadline. Even when they do they inevitably miss something or other that's pretty serious.

      Testing should not be scheduled because no one knows (or can know) how long it will take ahead of time.

      That does not make testing instantaneous. Fred Brooks, author of The Mythical Man-Month (which is published in book form along with other essays of his; I strongly recommend the book) advises that a full one half of project timelines are typically spent on testing, whether testing time is scheduled or not. I have observed this myself to be a pretty good heuristic.

      Another mistake people make is not having a test plan, or if they do, not scheduling test plan development! I have found that a test plan can be derived from a thorough design document fairly easily.

      Testing should not be scheduled because the customer will get mad and possibly go with another bid.

      The best of the three argumento IMO. Try to explain to the customer that testing is vital and that other vendors will spend just as much time on it; it's just that *you* are being up front and honest about it. If that doesn't work try not specifying a delivery date due to indeterminate testing time. Last resort, put an "I told you so" clause in the signoff document.

      I've rambled too long. Don't forget testing time!!

    A reply falls below the community's threshold of quality. You may see it by logging in.
Re: How to calculate development time?
by tachyon (Chancellor) on May 31, 2001 at 16:19 UTC

    This seems like a "how long is a piece of string?/how many fairies can dance on the head of a pin?" question.

    I find the following formulae useful for calculating development time:

    $dev_time = ($deadline-$today) + $several_weeks; $dev_time = $developer_estimate*2 + $fudge_factor; # note $fudge factor is always a positive integer! if (use strict && use warnings) { $dev_time = 0.5*($dev_time); } if ($no_planned_approach) { $dev_time = $developer_estimate * (2+rand(5)); } if (#!/usr/bin/perl -T and use CGI.pm) { $security_nightmares *= 0.01; }

    Yes these formulae are silly but experience seems to bear out these basic principles.

    The first step is to define your project in precise terms, what do you want your site to do, why, and how...This is vital and will save untold hours in the long run.

    The answer then depends of course on your knowledge and experience, your spec, and how little you will actually have to code by adapting available modules/code to your needs.

    One problem with modules is that they *can* have a steep learning curve, so you should possibly stick with the basics. CGI.pm will be essential.

    One approach is to try to find a script that fulfills most of your criteria and then adapt it. Sadly you tend to get what you pay for but of course there are so many exceptions to this rule...especially in the give and let live Perl community.

    CGI.pm is the basis from which to start. It can generate HTML for you as well as get CGI input. It allows file upload so users could upload say HTML CVs.

    Search engines to index your data can be complex to code. In a pinch you might consider one of the free site search engines. I use http://siteLevel.whatUseek.com for a cheap and cheerful search, on shoestring sites. You can customise the search interface and the output template and it is free for ?1000 pages I think. To see simple example in use check out http://www.fg3.com.au/home.htm. This site (still under construction of course!) took 6 hours to generate, graphics, search, etc. Try the search, and then look at the 4 lines of source HTML that are required. If you have CVs and Job Ads as *.html pages that can be spidered this will solve the search requirement in under an hour. Downside -> ads on result page. Hint, edit the template and dump the adds to the bottom - if you remove them they get replaced at the top, but if you move them to the bottom all is OK.

    Consider employing a perl consultant to set up at least the basics for you to tweak

    It should not be a problem to get modules installed at your ISP, often in your local bin where you may or may not be able to tweak them. When it comes to CGI most ISPs are aware of the security risks and have varying policies. Ideally you will want to be able to freely upload your script during testing but this requires that your ISP trust your code, or be non security concious. This is a real issue when it comes down to getting your script running. If you have to get your script vetted (often for a fee!) with each change the process becomes very slow. This *should* influence your choice of ISP. PS You may not want your prodution code to reside on an ISP that let's anyone edit their CGIs but.....

    Finally (whew got a bit carried away here) if you want to simulate CGI, have relatively little experience, are familiar with win32 then perlbuilder 2 from solution soft is a very useful program. Trial copy at www.solutionsoft.com.

    Good luck, I'm sure you will have fun discovering the power of Perl, the greatest CGI lang on earth. If you have specific code problems that the Monks can sink their teeth into you will often find help here. Homework and writing your code for you is a lottery and you know what the odds are in lotteries!

    tachyon

      Thanks for all the feedback from all of you.

      I know it's a very general question, and impossible to answer, but you are monks after all. I really appreciate the speed of replies.

      I have not decided on an ISP yet, but are looking for something in the UK with full CGI and MySQL support. at the moment Internetters seems like a good bet. I would appreciate other possible ISP's. I have worked with Internetters before, so that is probably why I would go with them. They hosted a site which I developed for my day job and it seems to be OK.

      My biggest problems are:
      1. I only have FTP access, no Telnet.
      2. I don't have access to the Apache error files

      Up till now it hasn't been the biggest problems since I developed everything on a development machine and tested it before I uploaded it. The development and hosting environments aren't exactly the same though, which creates a problem now and then.

      I am however a bit worried about MySQL development on a remote host with no error logs. Is there any way I can see what is not working without the ubiquitous Internal Server Error message?

      My next question regards searching. If I want to search through the MySQL DB, is it going to be as easy as:

      SELECT * FROM jobs WHERE creation_date < $search_max_date

      or do I need something like siteLevel.whatUseek.com that you suggested?

      It's been a while since I did any SQL, but I'm sure that it's not going to be the biggest problem.

      Here are a list of what I think needs to be done: (My solution in brackets, comments would be appreciated.)

      1. User regestration ( .htaccess files, MySQL table for user profile from HTML form, basic CGI)
      2. Employer registration (same as 1)
      3. Upload jobs (MySQL table for jobs, linked to employer table. html form, basic CGI)
      4. Remove jobs (same as 3)
      5. Search (???, MySQL SELECT queries?)
      6. Credit Card Billing for employers (No idea..., maybe use something like NetBanx, help would be appreciated.)

      thanks for everything
      -Siddartha

        I am however a bit worried about MySQL development on a remote host with no error logs. Is there any way I can see what is not working without the ubiquitous Internal Server Error message?

        Yes;
        use CGI::Carp qw/fatalsToBrowser/;

        This prints nice error messages to the browser; however, if your script dies before compiling, you just get a (fairly) unhelpful "compile error."

        If you don't have a command line, maybe you can do your developing on another box somewhere and then move everything to your host? I personally could not bear to write any decently sized script without the debugger.

        i would wholeheartedly recommend Positive Internet (www.positive-internet.com) if you are looking for an excellent Perl and Mysql hosting company in the UK.

        Having used a more well known host before Positive, i can state that the folk there are v helpful and the service is outstanding - they have even installed perl modules that they didn't have but that i wanted to use, which has been great.

        Tell them iain sent you if you go with them - they have a nice referral scheme too ;)

        also, i can't seem to locate an email address for you if you are looking for help via mail - anyone help?

Re: How to calculate development time?
by arhuman (Vicar) on May 31, 2001 at 16:26 UTC
    I use the following rules when I try to estimate developping time :

    • No serious estimation without complete specs
    • Don't assume that when you do several times the same thing you'll do it faster
      (In fact you'll probably get bored and it will balance the speed gained by experience)
    • No real task should be made professionnaly in less than half a day
      (Real analysis,Test, Documentation...Not only implementation)
    • Try to estimate the time to do each small task, sum up, then multiply by 2
      (test, client interaction, integration...)
    • If I don't have total control I can't produce correct estimation
      (state it but try to give an estimation anyway :
      "Provided the ISP had mod_perl properly configured it will take X hours, add X hours if I have to only use PHP...")
    • Any addition to the specs will change the estimation
      (even if they take something out, beccause your prior analysis and possibly data structure may be now inadequate...)

    Following those guidelines have saved me several time.
    "Only Bad Coders Code Badly In Perl" (OBC2BIP)

      Try to estimate the time to do each small task, sum up, then multiply by 2 (test, client interaction, integration...)

      You can get better at that factor. Keep a backlog of your estimates and the actual time you actually needed. This way, you can adjust the multiplication factor for future estimates. The factor will vary with many influences, including working environment, project scope, growing experience, business domain, quality requirements, and so on ad infinitum. Still, you will get much better estimates by recording your past performance, and extrapolating from that by way of an adjusted multiplication factor.

      Christian Lemburg
      Brainbench MVP for Perl
      http://www.brainbench.com

        You're obviously right...

        I just wanted to emphazize that whatever the time you estimate,
        increase it to be able to handle all the possible problems which may arise.
        (Furthermore I've whitnessed that almost everybody tend to be too optimistic when estimating...)

        You won't lost a penny if you are thru before the deadline.
        But being late is always VERY unpleasant.

        "Only Bad Coders Code Badly In Perl" (OBC2BIP)
Re: How to calculate development time?
by shotgunefx (Parson) on May 31, 2001 at 15:37 UTC
    If you are going to work on it only a few hours a day, I find that a huge time waster for me is getting back up to speed where I left off.
    (Often I'll work a day or two straight on something complicated otherwise I waste half my time trying to remember where I was).

    As far as calculating time (and cost), the job detail is pretty vague. Sort of like asking "How much is a car", there's a whole spectrum from the Yugo to the Bentley. I'd be very careful to state everything that you are and are not contractually obligated to perform and guarantee.

    When I first started working for myself, I played it fast and loose. I've done a lot of different types of projects and my <nobr>"Guesstimates" ™</nobr> have always been really close... until.. I had a client that expected our 5 figure software to do everything that there aborted 7 figure software couldn't do. Do to the lack of a rigid spec, it was a painful ordeal. Luckily they had a burn rate on the order of a super nova so the problem resolved itself.

    I know this doesn't really help come up with an estimate but I hope it gives you a bit to think about.

    Whenever I am quoting a job, I mentally map it out into the base components and actions and think can I do this easily? Have I done it before? If the answer to the latter is no, I do a proof of concept real quick to determine the complexity.

    When all else fails, come up with a reasonable and realistic guess of cost and time frame and then double it. :)

    -Lee

    "To be civilized is to deny one's nature."
Re: How to calculate development time?
by Abigail (Deacon) on May 31, 2001 at 19:44 UTC
    I have decided that the cheapest and best option would be to use Perl and MySQL, hosted by an ISP.

    I have a serious problem with that. I won't argue the cheapest part, but if I were to give you the assignment and you came back to me saying Perl, MySQL, hosted by an ISP in such a way you cannot control which modules are available is the best solution, I'd thank you and hand over the assignment to someone else.

    First, let's make the assumption the company actually cares about the website. If not, well, who cares? But if they really care, do you really believe it's in the best interest of the company to host in on an ISP that gives you very limited control? I seriously doubt so.

    Secondly, let's assume the company not only cares for the website, but also about the data. If you care about data, MySQL should not even *be* an option, let alone a "best" option. Check for instance http://openacs.org/philosophy/why-not-mysql.html for some reasons why MySQL should not be an option.

    -- Abigail

      Thanks for your comments.

      I will seriously consider other options, if I knew what they are. The client is a startup company, so it doesn't have a huge bunch of capital to spend on the site. They also consider advertising to be the one thing to spend most of their money on, not the development of the site. Obviously the site is important to them, or they wont spend such a lot of their startup capital on it. The site will basically be their means of income, so it is a crucial part of the company.

      So, why did I decide on Perl and MySQL?

      Firstly because I know Perl (a bit at least, and I really don't want to use ASP), and have used MySQL before for other stuff.
      It is the cheapest option.
      I don't think the client can afford a dedicated server at all.
      That basically leaves an ISP.
      If anyone knows of a ISP that is not too expensive and lets you install Perl modules etc., then please tell me.
      Regarding MySQL: I don't think that the posting of a job on a jobsite is the kind of data that warrants using Oracle etc. I will not use MySQL at all for billing purposes, at the moment I am looking at using an external service like NetBanx, but would appreciate other options.
      I am basically looking for a Database that will store data that I can search. A flat file is too much hassle to use. I do think that MySQL is therefor the best option for me.
      If however I find an ISP that has another Database and is not too expensive, I would consider it.

      I would appreciate it if you can give me concrete reasons why Perl and MySQL would not be a good solution, even if it is hosted on an ISP.

      -Siddartha

        Firstly because I know Perl (a bit at least, and I really don't want to use ASP), and have used MySQL before for other stuff. It is the cheapest option.

        That's not much of an argument, is it? Sure, you don't pay license fees for Perl nor for MySQL. But you don't pay license fees for Java, C or Python either. Nor for Postgres. Note, I am not advocating you should use any of those. But "I have used X and Y, and I don't want to use Z" doesn't make X and Y the cheapest solutions. Development, maintenance, modifications all take time - and people working on them will want to get paid for it. However, I wasn't argueing price.

        I don't think the client can afford a dedicated server at all. That basically leaves an ISP.

        Interesting. You don't even know whether the client can afford a dedicated server, yet you conclude using an ISP is the best option? Sorry to say, but that doesn't sound like you've done much of an investigation. However, there's nothing wrong with using an ISP. But it is wrong to tell your client his best option is to use an ISP that doesn't give you much freedom. If you can't even install modules of your choice, your client is much better off with an ISP that does give your client at least that.

        I don't think that the posting of a job on a jobsite is the kind of data that warrants using Oracle etc.

        Did you ask your client how important they think the data is? Can they afford losing the respect of their customers (the people using the website)? If your client doesn't care about losing the data (resumes, etc), then why are they collecting it in the first place? Is your client planning to make money (directly, or indirectly) with said data?

        I would appreciate it if you can give me concrete reasons why Perl and MySQL would not be a good solution, even if it is hosted on an ISP.

        I didn't say I objected to using Perl. There might be reasons not to use Perl, but your article didn't say (if the application is going to be developed and maintained by a team of 40 Java jocks with no Perl experience, Perl is likely not to be a good choice). I did, however, post a link to a page giving a multitude of reasons not to use MySQL. If those aren't concrete, what reasons are?

        -- Abigail

        Siddharta wrote:

        The client is a startup company, so it doesn't have a huge bunch of capital to spend on the site. They also consider advertising to be the one thing to spend most of their money on, not the development of the site. Obviously the site is important to them, or they wont spend such a lot of their startup capital on it. The site will basically be their means of income, so it is a crucial part of the company.

        Company decided to be cheap on basic mean of income, and instead to spent most of money on marketing? Looks like "get-rich-quick" scheme to me.

        If you decide to go for it, at least get agreement to be paid step-by-step. Protect yourself before possibitity that company will go belly up before you got paid.

        For me is very hard to imagine a situation where I can use database without solid atomic transactions and rollback. If MySQL cannot support rollback, I will avoid using it for important production data even if paid to do so. And in no case for payment & Credit Card processing. Are you looking for troubles?

        You are taking rather big risk, I hope at least your award is worth it. :)

        pmas

        I've used MySQL for websites that get millions of page views involving up to hundreds of millions of queries per month without ever having a problem.

        I should repeat that for emphasis: I've never had a problem. No lost data. No mysterious crashes. Nothing except good performance and reliability. (Performance was about 9 times faster than Oracle by my benchmarks in the application I was running at the time it was ported to Oracle.)

        The link that questioned MySQL as an adequate database and questioned whether it was just waiting to fall apart at the seams because it doesn't have atomicity or rollback obviously was put together by someone totally unfamiliar with its use in a production environment and who seems like they were more concerned with communicating their knowledge of esoteric (but good to know!) database concepts than reality.

        Of course, it doesn't have as many features as Oracle or other large commercial databases, but it sounds like for your needs, it's just what you need.

      After reading this reply, and being a mySQL user, I naturally went to read much of the refered to page (it's very long).

      It's a very interesting sequence of posts and responses, not so much because it was in any way conclusive on the "Not MySQL" front but rather because its a good lesson in how things change. It's a conversation that started a year ago and still continues, and shifts subtly as the various systems it discusses have changed over that time. It's also a good lesson in how inteligent people can have radically differing opinions because of the the questions they think are important to ask, and the problems they think are important to solve. I highly recommend it, but again not because it "proves" that MySQL shouldn't be an option.

Re: How to calculate development time?
by pop (Scribe) on May 31, 2001 at 18:59 UTC

    Here's my personnal experience. I have developed two years ago a similar site. I was student, perl newbie, apache intermediate, SQL newbie, XML very-newbie, good knowledge of linux etc...

    I had some technical knowledge in SQL, perl and so on, but no concrete experience. My colleague and friend (student too) was at the same level than mine.

    We've spent 3 months full time plus 2 months coding during the night. And we went with a mod_perl (Apache::Registry) site, XML & SQL data, Template system (HTML::Template)

    This site was a rewriting of an older one (mod_cgi + msql + print $html; + no cookie + no session :)))
    Finally, the site major features were : Job posting, Resume posting, candidature system, application letter, Pro directory ...

    I hope this help you to figure out the developement time, but I guess that it depends on plenty of details... So the other posts should be more precious than this one :)
    Have Fun!!

      Thanks a lot.

      The other posts does give me a better idea of what to do, but it is really nice to get some actual concrete ttimes. I do really hope that I can do it quicker than you did.

      I am doing it on my own (Mad, I know :)). I know CGI scripting quite well, even though I have not used MySQL with Perl at all. I know basic SQL query syntax, enough to update, delete and select data from tables. I don't know if I'm going to use XML at all, not yet anyway. I know a lot about linux, but unfortunately only have FTP access to the site.

      I am writing this from scratch.

      I have about 4 months to do it, and can only work on it part time. I basically need to know if I am going to crash and burn?

      I would really appreciate any specific comments on what to look out for when developing the site.

      It will be a very basic system to start of with;
      Job posting, resume posting (both as Form data), and search for current jobs/employees. My biggest problem at the moment is that he wants credit card billing for Employers when they post jobs, and I have no idea how I am going to do this. At the moment I am thinking of using someone like NetBanx to handle this for me.

      Once again thanks for all the wonderfull fast replies.

      -siddartha

        I wouldn't take credit cards unless you can control the server. Taking credit cards is a big responsibility, and you will have a hard time proving that you did it correctly should anything go wrong. Although I am used to US liability laws (and am therefore paranoid about such things), I wouldn't want to be the one who got sued because someone hacked into the ISP and set up a credit card sniffer.

        I set up a credit card system and found it to be painful, because I had to deal with:

        • Learning how credit cards really work. This isn't as simple as you might think. There are many issues that I can describe if people are interested.
        • Dealing with the US Department of Commerce denial list. They keep a list of bad guys that I wasn't allowed to sell to.
        • Security reviews by a guy who was long on paranoia and short on real knowledge. He insisted that I implement a really twisted system that had performance problems.
        • An antiquated order processing system in the company.
        • There were many more...
        Once the site is running, you need one or more people in a production and support role. This is the area that many people don't think about at all, and it makes many sites fail quickly after deployment.

        Given the level of experience you have described, the tools that you have available, and your time frame, it sounds like a crash-and-burn to me. Debugging problems will be extremely difficult, especially with your credit card system, which is difficult to prototype. For example, say your credit card transaction times out. How will you solve the problem?

        It should work perfectly the first time! - toma

        Hi Siddartha,

        I would /msg this but it's going to be too long, sorry everyone for this ad-space!!!

        Apart from Netbanx, you could also try WorldPay. They claim you can get set up with them in around 48 hours - (in reality it takes a little longer because of form filling). There are the following benefits:

        • You have the option to accept payment in multiple currencies
        • It's very easy to integrate, although the look and feel is not very customisable.
        • You don't ever store any credit card information so you don't have to worry about cc info being stolen from your server.
        • They handle repeat billing for you automatically - e.g. subscription fees.
        • web-based statements and refunds facility for the merchant
        There are loads more options in the UK for credit card billing such as securetrading - but that is much more work to integrate and get set up with them.

        On another note, you'll need to decide about who owns the intellectual property, copyrights, etc. Plus, make sure you don't give them the site without getting paid. If this is your first job, then you might fall into this trap - because it's really easy to think that you need to be very accomodating to the client without covering yourself.

        A client of mine developed a job site for someone and has only received about 10% of the fee even though the site was finished about 9 months ago.

        Bottom line: try and get everything in writing, if possible arange for some payment up-front or after the first phase is ready for the clients to see.

        $code or die
        $ perldoc perldoc

        In fact, I think that a very basic system for beginning could be developed quickly. As I said, my site was an enhancement of an older one, so we have spent about one month wondering how we could add the required bunch of features to the older site. (In fact, the resultant site was quite complex, featurefull...)
        So, you should gain one month :)

        You should gain more than one month since we had to learn the usage of CVS, XML::Parser and how to use Apache::Registry. More important, we had to find these tools. It's not obvious when you're just-a-newbie, and you don't know how-to-do and why =)

        Nevertheless, it takes time. A lot of time, since there are always problems, and your boss/client always add some feature enhancement... (that's why it's important to have good written specs and a good contract, as it has been said in other posts)

      Here's another note from personal experience. I've also helped developed a similiar system, using Perl and Postgres. I think our project was toward the small end. The features included:
      • Users could set up accounts and update their personal information.
      • Admins could add an update jobs and basic organizational information
      • Public could browse through jobs by category.
      The site was small enough that we didn't need a search engine, or mod_perl, or XML stuff. There were 12 fixed organizations that could post, so we didn't much interface to manage that end. I believe we spent about 50 billable hours on that. (factor in that we already have strong Perl and SQL skills and have full control and access to the hosting environment.)

      I agree with all the other folks that getting a clear spec with as many details as possible before you start is a Good Thing. We usually develop a functional mockup first, using static HTML and JavaScript clickthroughs on the forms. When that is refined with the client then we use that as the core of the final spec, and make a refined estimate with it.

      -mark

Re: How to calculate development time?
by bikeNomad (Priest) on May 31, 2001 at 19:06 UTC
    After a recent unpleasant experience, I can add an important and time-consuming first step:
    <bl>
  • write a contract and get it signed!
  • </bl>

    Nolo Press has a good (downloadable) book called Consultant and Independent Contractor Agreements that I used (too late!) to produce a good contract.

    Don't do too much work before you get a contract signed, because without a contract it can be harder to get paid when things change. In my case, I put in about 80 hours of work (including contract preparation) and then found out that the reason they were so slow getting the signed contract back to me was that they'd hired someone in-house to do the job I was contracting for!

    If you have to do a lot of work up front to estimate (which you may have to), try to get paid something up front.

    And be careful that you specify the system in terms of behavior rather than in terms of any particular technology (because you don't want to commit yourself right now to a specific implementation).

Re: How to calculate development time?
by fs (Monk) on May 31, 2001 at 17:17 UTC
    Development time is one of those bits that you're almost always going to get wrong, as development almost always takes long than you think. For a good read on the topic, check out Fred Brook's Mythical Man Month. It's been around for over twenty years, and is still considered one of the best books on the topic.
Re: How to calculate development time?
by sparky8342 (Acolyte) on May 31, 2001 at 17:55 UTC
    With regards to mod_perl, it's unlikely that you'll be able to use it on a shared hosting environment. You would need access to apache's configuration file, for one thing, which would give access to all the virtual hosts on the machine being used by other customers.

    Dave
      Do I still need access to the config file id I can get the ISP to install mod_perl ?:)

      If they add:

      LoadModule perl_module modules/libperl.so
      AddModule mod_perl.c

      and add:
      SetHandler perl-script
      PerlHandler Apache::Register
      Options +ExecCGI

      in my Virtual Host setup, would I need access to the config file for anything else.

      I don't know what the chances are of getting them to do this, but I would still like to know if the is all that is needed for me to use mod_perl in my HTML files.

      Thanks
      -Siddatha

        I think it's unlikely that they'd go for it.

        As well as the above, they need to recompile Apache with mod_perl. It's also conceivable that you might want to use different apache handlers later on, requiring changes to the configuration file.

        There's also security problems, you could potentially crash apache and bring down all the sites on the machine, etc.
Re: How to calculate development time?
by Starky (Chaplain) on May 31, 2001 at 21:20 UTC
    This is not so much a comment on calculating development time, as many of the other monks gave considered responses obviously borne from experience.

    This is a comment about your ISP.

    I would be more aggressive in finding an environment that is suitable to you. What kind of work can you do if you are constrained to a machine you have no control over? (FTP but no telnet? Noone should work under those conditions!) You need to find something else.

    I would recommend something like rackspace.com that will simply set up a cheap Cobalt for you (rental if you can't afford to buy one) in a colocation facility with Linux, Perl, Apache/mod_perl, MySQL, or whatever, then give you root.

    They charge by the month, so if the project is worthwhile, you'll be able to pay your bills. If not, you just move on.

    I would guess that what you pay over hosting off your ISP will be more than made up in the time and cost savings you get by having an environment you've got some control over. And you'll be a much more sane individual at the end of it all.

    And if your client can't afford to provide you with at least an adequate and production environment, then they shouldn't be undertaking the project in the first place! If they aren't bright enough or experienced enough to realize this, you need to communicate it to them. It will save you both alot of heartache.

    Hope this helps!

Re: How to calculate development time?
by Henri Icarus (Beadle) on Jun 01, 2001 at 01:49 UTC
    One thing I haven't heard anybody discuss here, which is my favorite method of calculating development time, is to actually just start doing the work, and keeping track of the time you spend, before a contract, before anything. Then, after some period of time, look back on how much you've accomplished think about what's left to do, and you'll be able to offer a pretty good development time estimate.

    Call me crazy, but this method works on lots of levels:

    1. Most of us are in this business because we have fun coding, so the time you spend (even if you don't get paid, or have to throw it all away because design decsions change) will have been fun.
    2. You learn tons each time you just start doing the work. I've written complete look-alikes of things like HTML::Template before I knew it existed. Of course it looks like the folks who wrote that did a much better job than I did. So what? Now know exactly what it needs to do, and I'm in a great position to evaluate it and to modifiy it.
    3. It's probably the method by which you'll get the best actual measurement of how long it's going to take. For smaller projects, you get it 100% right, and you bypass the "design specs" section all together. How many of you have had the experience that your clients only half know what they want, and part of what they are hiring you to do is tell them what they want? What better design spec is there than a working project? As long as you have good coding practices modifiying things to the final design (which is never the same as any "design specs" anyways) isn't that hard.
    My direct response to you Siddartha is, unless your mortgage and kids are on the line and you won't like what it does to your life-style, then just jump in and do it. Damn the torpedoes. Definately do use CGI.pm, and probably use CGI::Application and HTML::Template these will both save you lots of time in the long run despite modest learning curve issues. Whether you use MySQL or PostGress or flat files, big deal, just use the perl DBI and switch later once you get things up and running.

    Responses?

Re: Choosing your ISP
by Reverend Phil (Pilgrim) on May 31, 2001 at 19:34 UTC
    Hi chief.

    While everyone else seems to have expressed all the requisite concerns about estimating without specs, as well as some pitfalls to watch out for when estimating, I have an idea on an ISP that might be useful.

    Bearing in mind that I have not used them, but merely came across them accidently and read much of the info on their site, (and therefore can't wholeheartedly recommend them) take a look at PowWeb, as they provide web hosting with cgi access, Perl and PHP scripting, MySQL, and a bunch of other things that you don't often find on low-maintenance cheap ISP hosting services. Let me know if you end up using them or not, and why. I'm curious about their absurdly low price for what they offer.

    I just sent them an email asking what modules they have available for Perl, if there is a way to request additions, and if they support mod_perl in some fashion. I'll reply to this post if I find anything out.

    Good luck,
    -=rev=-

Re: How to calculate development time?
by Anonymous Monk on Jun 01, 2001 at 08:27 UTC
    I am offering an entirely different spin. Since this is going to be a MySQL based jobsite, why reinvent the wheel? There is a wonderful software package called ForwardSQL which is essentially a suite of scripts written in C that I'm sure can do everything you need. I would buy the package and pass the cost along to the client. Your time will be spent designing the site and NOT writing code. The URL is dbwww.com. I have no affiliation with them, but offer this suggestion as I have used this software in the past and it is quality code. You may need to write some custom apps depending on the nature of the project, but by and large the ForwardSQL program can handle 99% of your needs. I have discovered that by using their code, my development time is quite rapid and has allowed me to win a number of project bids because of the decreased development time. Just my $mytake=$thoughts*.02; $opinion=sprintf("%.2f",$mytake);
      Thanks for the suggestion.

      I am seriously looking into this. The only problem will be that I will need a dedicated host, or an ISP that gives me telnet access. The cost of the solution is also not that much, and the increase in productivity should more than make up for it.

      Thanks again, I have received a lot of usefull information from everyone, but this is probably the most usefull reply to my specific needs.

      -Siddartha

        Hi, I wrote a pretty detailed response but Netscape crashed.. so here are the highlights.

        - Mysql is fine for this job, it will take longer for Oracle which is overkill. I've used both and Abigail's comments notwithstanding, I can tell you your choice is fine. I read most of the document she provided and while there are good points there are also a lot of people with great experiences on MySQL and I find much of it outdated FUD particularly with regard to the needs of your project.

        - You must have telnet access to stay sane now and in the future, you might need to look at the apache error logs or maybe compile your own perl modules some time. Rackspace is nice, I have not used them but have been very impressed with their responsiveness at night. ssh is better than telnet if you can get it. I recommend linux with RAID 5 disk array.

        - Unless you must do clickpath tracking or very high volume accesses, don't use mod_perl on this project. It is a lot of fun but you need to be able to restart the server and it takes a lot longer. There is not a ton of documentation compared to the number of issues that come up with it, so experience is important.

        - Don't do credit cards in the beginning. It is complicated and other companies and laws will be involved.

        - As for project schedule, you may have trouble on this project if you allow the client to eat up your development time with figuring things out. Development time starts after the spec is signed.

        - You need to get a dialogue going and confirm your understanding of what they are saying. The client has to participate in developing a detailed specification, and must also be coopted to promote success of the project by the following activities:

        - Signoff on screen mockups and detailed functional descriptions before development

        - Signoff on delivery of parts, if you can handle that.

        - You make weekly risk assessment reports ( a number 1-10 may be good) and use "that increases risk" and "decreases risk" when new functionality or deadlines are mentioned. The idea is that you do not want to get behind the eight ball trying to defend yourself against an unrealistic deadline from feature creep or some other problem that may crop up.

        - Check out some of the ideas of XP programming philosophy, it has old and new ideas and may give you some ammunition if you use some of them.

        - It will probably take 2-3 times as long as you think. Keep a schedule with expected man-days per task, including testing on your side, bugfixes, testing on their side, content delivery, whatever. Actually content delivery is in addition to that. The variables you generally have include the number of people on the project, the budget, the time until delivery, the quality/robustness of the code, and the SCOPE. Scope means how much you are going to do. It is easiest usually to change scope so you must get the client involved so they do not make it difficult to change scope.

        - There was a very good recommendation above that you look at how your man-days projections match reality and then multiply the difference as a time stretching factor across the entire project. I didn't believe it when I first heard this but it is true.

        - Make different phases of the project. The first phase is consulting which you should get paid for, it involves making the spec and figuring out what they want down to screen mockups (line and text is fine). Also a designer is probably going to be involved. Make sure of what you are responsible for, including do you write HTML. Supported browser types may be another thing to make sure of up front.

        - The different phases will make it possible to move functionality to a later phase if you need to. Have the client set priorities (no, they can't say 100% or nothing) so that you can identify what core functionality must be perfect by launch, what is best effort, and what is later phase. Then you can get them to put feature creep functions into the next phase, or get them to change other variables like schedule or maybe remove a different function.

        - Get feedback as soon as possible from other people in organization so you don't get nailed at the end. Also try to decide as much of the software architecture as early as you can, preferably well before signoff on the spec.

        It looks like this is a dangerous project if you don't think of some of the above things. You will have to help your client figure out what he/she wants, and you are only working part time while there is a learning curve for some of the things you will be doing. I would say give yourself a large buffer, start the clock after signoff on the detailed spec, and get the client involved in the process so they can see what will happen if they become unrealistic about things. Don't handicap yourself in the beginning by agreeing to a lump payment at the end (especially if you are meeting milestones and have a consulting report deliverables in the beginning), and if schedule is really important then discuss the possibility of budget changes to add another guy if you really need to at some point.

        That's all for now, good luck!

        Have you looked at Positive Internet? Full ssh access, mysql, mod_perl, the works. You can add your own modules, but normally you're better off asking the very helpful tech support guys to add the module to the server so everyone can use it.

        From experience on a cgi/database heavy site (http://www.moonfruit.com), mod_perl is *essential* if you are anticipating scaling to larger numbers or users. "use strict" and you'll have no problems with mod_perl.

        MySQL has limitations, but is cheap. No transactions (OK - no "normal" transactions...), no stored procedures, no sub-selects (ie no "SELECT * FROM users WHERE user.id IN (SELECT userid FROM admin_users)". But it *is* free, and it *is* included in many web hosting packages

        You won't get a decent site done in your spare time in four weeks. There are issues you haven't thought about yet. For example, have you thought of including an admin interface for the people running the site to delete/edit users, get stats, change passwords etc? Of backing up the data and being able to recover it?

        Clients, eh? Who'd have 'em...

Re: How to calculate development time?
by stuffy (Monk) on Jun 03, 2001 at 04:29 UTC
    instead of starting from scrach, maybe something like what is avaliable at everything development with some modifications could help you out. remove some features, add some. It might save some time by giving you a start.

    Stuffy
    That's my story, and I'm sticking to it, unless I'm wrong in which case I will probably change it ;~)
Re: How to calculate development time?
by Johnny Golden (Scribe) on Jun 01, 2001 at 01:33 UTC
    There is very little I can say that hasn't already been said as you received a lot of great advice on this. The one thing that I can offer is that when giving project completion dates, is to qualify them. Once you figure out the spec, you can give a date but specify that that will be the date, provided they make no changes. I have found too many times that once a project is underway, the client wants to make a major change in how they want to do business based on something they read or something their kid told them at dinner. Thank god that Perl is so flexible!
Re: How to calculate development time?
by Anonymous Monk on Jun 04, 2001 at 00:10 UTC
    I echo what other folks have said above.

    It strikes me that you might have to be a bit of a salesman to your client: Tell them that using some sickly ISP will slow the project. I've hosted sites at (and been reasonably-pleased with) Rackspace since February 2000; having that much control over your environment is invaluable. You'll thank yourself when you're running into problems.

    Also, this might be heresy in some circles, but you might want to take a look at Slash. I've never looked closely at the quality of the code (I've managed to change what I've wanted without problem), and it has a bad reputation for clarity, but it is a working system that you might want to borrow from. Remember that it's GPL'd.

    -Declan (declan@well.com)

      Thanks for all the comments.

      I have drafted a proposal, splitting development into phases. I have also requested to use rackspace for hosting. I am currently waiting to hear what the outcome is. I am also looking at purchasing forwardSQL as suggested by an anonymous Monk.

      If all goes well I will have enough time to really sit down and play with Perl, instead of frantically trying to debug everything.

      Slash is interesting and I will look at it.

      I think my biggest problem right now is to decide what exactly to use.
      Will I use HTML::Template, Mason etc. ?
      will I use mod_perl?
      Will I use Mysql? (I still think I will)
      If I do get a dedicated server to play with, what other possibilities are out there to do this.

      At the moment I know that I am going to use Perl, CGI.pm and DBI with a database (probably MySQL).

      any comments would be appreciated.

      -Siddartha

Re: How to calculate development time?
by tame1 (Pilgrim) on Jun 04, 2001 at 14:06 UTC
    Rather than leaving yourself screwed like that, why not deal with a
    hosting site that will give you ssh access and allow your own
    modules? Try www.mhost.com. You can do pretty much what you
    want there, within reason. Plus I personnaly know that all
    of the Perl modules you could want are installed - I made
    him install em. And they run MySQL.

    What does this little button do . .<Click>; "USER HAS SIGNED OFF FOR THE DAY"
Re: How to calculate development time?
by diakonos (Hermit) on Jun 06, 2001 at 17:03 UTC
    One of the most difficult things to do when contracting is understanding the wants of the client. Most people are not thinking that they need to explicitly detail what they imagine the project to be. I think they somehow feel that you can magically read their mind.

    With this in mind you need to do everything in your power to understand exactly what they want. This means not only getting the "details" in writing but also spending time asking the what if's...

    Try to get the thing working in your mind before signing any contracts. Also, most contractors are very comfortable with the language before they dive into a major project. I feel by what you have stated that there are too many unknowns at this time. It seems like it is diffcult to find ISPs who will let you have access to Perl or a cgi-bin directory these days.

    Most contractors have written so much code that they have their own library of subroutine skeletons and modules that they are very comfortable meeting a project deadline. I am not saying that the new programmer will not be able to do it but the contractor new to the language should allow a forgiving buffer time.

    The bottom line is KNOW what your customer wants and understand ALL of your limitations and don't try to pull a rabbit out of your hat. Then it may be a little easier. Good Luck.

    Best Regards
    Doug
      spending time asking the what if's...

      It just ringed with me. It is amazing, how often it happens with non-experienced users: When they explain specifications, they explain common cases, but forgot to mention any irregularities, saying these cases are very rare and they did not want to confuse you more. As it happens, these irregular cases might cause complete redesign to meet these "hidden" requirement.

      I read somewhere that if God, when creating universe, will ask for sign-on on complete requirements, we will be still waiting to implement.

      Author proposed "Genesis approach" (RAD): To create something simple in just 10% of available time, covering maybe only 20% of needed functionality, get feedback from customers, and let it grow. This approach assumes you have ready-made tools and tested procedures ready "to hit the road".

      Still, IMHO you may consider this especially in your case, when you are not sure about tools to use. Explain your customer that it might be it may take longer to implement full 100% of what s/he has in his mind, but you cannot read his/her mind, and both you and s/he will have better feeling what needs to be done, what are priorities and what features are not feasible.

      Of course, if you can read minds, just go ahead and do it.

      pmas

HTML::Mason
by Anonymous Monk on Jun 04, 2001 at 03:15 UTC
    First, know that HTML::Mason does NOT need to be run under mod_perl. It's a bit nicer if you can, but it's still wonderful if you have to run under CGI. I'm doing this with www.globalview-intl.com, which is hosted at www.korksoft.com. Korksoft's hosting plans, by the way, are very reasonably priced, and they offer a lot of great stuff including Mason, MySQL, ssh access, and so on. They're also good about installing Perl modules at your request. Getting support can be slow, but it comes eventually and this drawback is outweighed by the rest. Viel Gluck, Dylan Oliver
      It is nice to code your web application using HTML::Mason. Its a framework which can be tuned to our requirements. Use of autohandler make us to establish MVC architecture. -Ramprabhu

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://84516]
Approved by root
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others wandering the Monastery: (6)
As of 2024-03-19 07:56 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found