"It has some initialization that used to be done in the constructor but it seems that this is generally frowned-on in the DI world" - that's not exactly true. DI is just about providing dependencies externally not creating them within an object.
So first thing fo you is to identify, within your singleton, what are it's dependencies and which are other object that it just uses to execute it's logic but they don't need to be created externally.
For example, say you have you OldSingletonClass which uses instances of C1 and C2 classes, that have their own dependencies. These should obviously be injected. But say that this class is supposed to make some HTTP calls and you implemented it with a some HttpClient and this HttpClient does not take any dependencies. I'd say that in this case you can safely create it in constructor (if it of course makes sense to keep single instance).
As for creating stuff in constructor vs Initialize() method - my experience is it is sometimes discouraged to do too much stuff in ctor because of different object inheritance issues, but other than that it's just a matter of styleLast thing, remember that DI does not require to use DI-container. It is often very helpful if you're working with tons of objects and some frameworks that "magically" create new instances (ASP.NET MVC controllers for example), but in case that you have, say, 20 objects, all created at application startup, I find it is often much cleaner to wire them together in code.