in reply to creating and managing many hashes

> I have currently done this using arrays over the 2 million pairs that arise from 2,000 products and it ran for 4 days before

I think your speed problems result from linear searches.

You may want to organise your data in a nested structure.

See perldsc And perlref

Like (draft)

$h_record = { date => '2018-01-01', product => 'apple', price => 200, quantity => 50000, }; $product{apple}{'2018-01-01'} = $h_record; $date{'2018-01-01'}{apple} = $h_record;

Please note (like others mentioned) that this approach can't compete with an SQL DB in terms of efficiency.

Those hashes need lots a memory where a SQL is managing an index on columns of the same table. (Think of all the hashes you'd need to easily look up year, month and day separately)

Also reading files to build the data structure will cause you overhead where SQL is persistent.

Cheers Rolf
(addicted to the Perl Programming Language and ☆☆☆☆ :)
Wikisyntax for the Monastery