Survey Title: About your test suite...

Page 1

Page 2

This page was only answered by people who don't have test suites

Of the five people who answered 'other' to why they don't have a test suite, their reasons were as follows:

1. Research - exploratory only
2. Cross-compiling to platform with no testing tools
3. still learning how to write them
4. New job where I'm pushing for testing, revision control, etc
5. mod_perl

Of the six people who answered "other" to how soon they plan to add a test suite:

1. When hell freezes over or "there are no more features that could be implemented any more"
2. someday
3. as soon as we get round tuits
4. Never
5. as I have time
6. Starting now incrementally

Of the 27 people who answered this question:

1. learn how to make a test suite...
2. Start with basic testing until I understand it better. Then move up to more complex suites.
3. start from the ground up. Looking at what is a good way to start.
4. Unfortunately planning and writing test take time and effort. When documentation consists of your ability to read badly written legacy code and management has a incredibly urgent/ninja rush button firmly pressed down on every change request (no matter how small) it is hard to introduce a culture of testing.
5. None yet.
6. Start with some of the more easily separated, more sensitive areas of functionality, and build from there.
7. To use a test suite and learn how to use one.
8. vague
9. None
10. Depends on the organization of the final code. Currently it is a mix of Perl and Java. Once the research phase firms up and we establish a clear direction, we will use a test harness as appropriate for the language.
11. I'm still learning test based development, I've read a few articles and tutorials, but just started a project that will have test suite from the beginning.
12. Put tests into my next project
13. none yet
14. Refactor code to make it testable. It's a website whose html is generated by print statements and subs return strings of html. So: subs -> data html -> templates
15. None
16. If the tools were available, I'd at least start testing my models as soon as we have an initial version of our current programs out.
17. add tests to existing code as I can; build new code with tests in place.
18. From-scratch rewrite of large parts is coming up; will have tests. Tests for legacy codebase not considered worthwhile despite regular "oopsies" when rolling out changes.
19. Haven't given it a ton of thought yet. We have a mixed code base of ASP.NET (VB.NET/C#), PHP, Perl, static HTML, JavaScript, etc and I'd like to either find something that can work between all of these or I need to take the time to get more familiar with various testing frameworks.
20. write testcases for running code and collect the TAP output for statistics. We need test in the runtime environment, at compiletime there is not much to test (ie: no db, and faking one is no guarantee)
21. Ridiculously, management pays some junior person to spot-check whatever they feel like checking at the current moment. They think this is an adequate testing policy.
22. slowly add unit tests as code is touched.
23. test for every fu**** function
24. Require tests of all new code. Start with some simple back-fitting of tests. Not sure how to sanely test the large DB queries we do. Not sure how to test the rendering in the front-end.
25. Start with (a) parts that are used most prevalently (b) parts that are self-contained Write tests for these and gradually work outwards. Adding tests for all new code.
26. After a code refactoring to allow for more testable chunks, will add in tests.
27. TAP

Of the 37 people who answered this question:

1. teach me how to make a test suite
2. new project
3. Time
4. The boss oking it. Several easily accessible tutorials with examples. Good papers on why testing is needed.
5. Do I use Test::More, Test::Harness, etc? I have lots of stuff to look at but I haven't sat down and figured out where to start, at least what is the current, suggested/approved way to start testing.
6. Tutorial on options and configuration.
7. Test::Simple where you can easily write tests in a manner similar to the expect programming language without needing hooks. Most test suites seem aimed at unit tests. Most of the shoddy code I deal with lives outside of modules (usually in web pages).
8. A formal block of general instruction and guidelines.
9. Convincing people of that it's worth the time spent.
10. More time to study.
11. a framework for testing CMS's af web-applications
12. Training and more time to devote to testing.
13. More programmers. :-)
14. Having more expertise, but I know I can't achieve without actually starting to use them.
15. A good tutorial.
16. more time
17. time from management to work on it
18. 1. A tutorial on how to test. 2. A "Test Patterns" like "Design Patterns"
19. More approved time by management
20. A complete rewrite of our codebase (testing is nearly impossible as is), and management that doesn't consider automated testing to be a complete waste of time.
21. Someone to come in with a Clue Bat and smack the entire chain of management around until they submit.
22. Tools for iPhone testing!
23. - The available book that everyone cites (Perl Tester's Notebook) doesn't work for me as a 'getting started' manual, so having alternate resources would be a great start. Maybe I just don't know where to look. - Time to work on it (this is a corporate issue). - Actual hands-on help from someone who knows what they're doing. I get so stuck sometimes, I dont' even know what to google. - A "testing dojo" at the local PM meeting...maybe I'll suggest that. :)
24. Having a boss that understands the benefit and cost reductions of implementing automated testing
25. Hastening the planned feature freeze of the legacy codebase...
26. Info on popular cheap/free testing frameworks for those languages/frameworks.
27. documentation on testing strategies. Most talk about ok(4+4,8) but that is not really a testcase IMHO, we also test response times as this is critical to user experience. We could just log this in the db, but I'd like to read about testing strategies for this using TAP (ie: the actual response time is irrelevant, it's either 'good' or 'bad' which can be determined fuzzy wise)
28. New management. :)
29. time
30. time
31. More time.
32. Tutorials with non-trivial examples.
33. How-Tos that show best practices for web site testing - preferably with Catalyst.
34. A better build system
35. Understanding testable code.
36. A raise and a bonus.
37. A down-to-earth howto.

Page 3

This page and beyond was only answered by people who do have test suites

Of the 25 people who answered "other" to the "What test framework" question:

1. Is what I plan to start with
2. None
3. Test::Class
4. Undecided
5. home-grown c inspired by Test::More
6. DejaGNU
7. N/A
8. N/A
9. homegrown
10. home spun
11. Test::More
12. schemeunit
13. N/A
14. Test::More
15. Test::More
16. Test::Class
17. Test::More
18. Test.pm
19. Test.pm
20. inhouse
21. Custom. Much simpler and easier than xUnit
22. Internal
23. Test::More
24. NUnit
25. Mine

Of the 8 people who answered "other" to the "primary language" question:

1. Would be perl, but you have no skip ahead option in the poll
2. N/A
3. Undecided
4. scheme
5. N/A
6. JavaScript
7. C#
8. Scheme

Page 4

Of the three people who answered "other":

1. What tests?
2. less than 1%
3. A tiny fraction, so far