I don't care about frameworks really. I think I am doing this from the ground up.

Intentional ignorance is a bad idea.

Everyone loves car analogies, so I'll make my point using one. Say you're planning on building a car (web app). Using a good modern web framework is like picking out an engine and drive train from a modern manufacturer, and then building the frame and body around it. In the end, the car is still your your car, with your design and your features and functionality. But, it has a well designed, tested, and high performing engine under the hood.

Building a web application from scratch means that you're designing your own engine before you can even start work on the car. You're recreating work that others have already done and is available for you (for free!). After you've spent a huge amount of time getting your engine running, you're finally ready to start building the rest of your application. Of course, the engine probably compares poorly to the other engines available, considering it's had less testing, less proving, and less performance tuning.

If your goal is to learn how to build a web framework, then you have a good reason to not use one. If you are writing a very small web application, then you have a possible reason not to use a web framework. If you're writing a web application and you know that there are no existing web frameworks that will work well with your application, then you have a valid reason not to use an existing one.

Note, however, all of the above options still require you to do some research and understand what web frameworks are available and what their use cases or functionality are. Beginning a large application without doing your homework is doing a serious disservice to yourself.