If you have large amounts of data, you should be using database indexes. This is how you add a database index to a DataObject in Silverstripe.
This is so easy to do, once you know how, its not funny, but it has taken me months to work out. I always thought there was a bug with the DataObject->toDropdownMap method.