I think that it is unwise to ignore even one percent of your visitors. When it comes to a business, loosing 3% of your customers may be the difference between success and bankruptcy court.
Building inclusive websites is a little extra work, but it's well worth it. Here are my rules (with grateful thanks to my hero Jacob Nielsen).
- Make your foundation simple and strong
In web terms that means USE TEXT.
- Include text alternatives for anything that isn't text
When you include anything other than text for content
always include a textual replacement for those who can't
see or hear it.
- Usability always wins
People don't care how cool something looks if that can't figure
out how to use it.
- Fail gracefully
When you have to turn someone away offer and
explanation as to why that can't enjoy something and alternatives
such as a phone number to call for customer support.
- Build for the customers technology
That means using templates that deliver custom versions of the
content best suited for their browsers. No matter how much extra
work it is for you, it's a lot easier that it would be for them
to adapt to your technology, and before they do that they will be gone.
Imagine what would happen if you opened up a regular brick and mortar business and placed a bouncer in front that turned away blind people. At best you would be forced out of business. At worst you could face legal penalties.
p.s. IMNSHO, This is a very OT question better suited for a web design list.