I haven't tested this, but I think this is a feature that's configured when you *build* perl.
From the Module::Build documentation:
This will generate HTML documentation for any binary or library files under blib/ that contain POD. The HTML documentation will only be installed if the install paths can be determined from values in "Config.pm". You can also supply or override install paths
on the command line by specifying "install_path" values for the "binhtml" and/or "libhtml" installation targets.
I don't have those paths in my Config.pm, and Module::Build doesn't generate html docs. If you want to turn it off forever, editing Config.pm is probably the quickest and surest way.
Otherwise, have a look at the installdirs and install_paths command line options. Something like
perl Build.PL --install_path binhtml= --install_path libhtml=
might just suppress generating html.