For consoles (and D3D12/Metal/Vulkan), you know everything you need to know to develop an allocation strategy. You know:
How much memory, of each kind, is available.
How big each kind of resource will be. Textures of a particular format and size take up X bytes, require Y alignment, etc.
Given such information, you can devise an allocation strategy for data uploading. Open-world games use streaming of fixed-sized blocks of data. Each sector of the game contains resources that take up X number of bytes, and therefore, artists have X number of bytes of storage to work with. It's their job to fit all of the stuff needed for that region of the game into that much space. There will also be stream blocks for larger collections of regions, or other data that gets shared between all game regions.
For other games, some of them load everything at level-load time. Some do hard-loads in the middle of "levels" (see many Source-engine games). Others stream things more dynamically, but even there, artists are restricted to using resources that total up to X number of bytes.
On consoles, such sizes can be known a priori, but with Vulkan/Metal/D3D12, these have to be queried for a particular piece of hardware. So in the latter case, you have to be more reactive to what's available, but all of the information is available to the user.
For other APIs/platforms (OpenGL/D3D pre-12), things are rather more complicated. Consoles and the lower-level APIs give you direct access to memory allocations, which gives you more flexibility. You can use a piece of memory for vertex data at one point, then put an image into it later on. Memory is divorced from its current use, so it's much easier for an application to choose how to recycle memory.
For GL/D3D-pre-12, things are difficult. A buffer object is a buffer object, and the memory currently associated with it cannot later be used as a texture. Well, it can, but you cannot explicitly ask that to be done. The most you can do is deallocate the buffer, allocate the image, and hope that the buffer's memory was recycled.
For cases where you're doing dynamic loading in fixed-sized blocks, you basically have to give your artists more limitations. Instead of limiting the artists by memory, you have to spell out the limitations by object So you'd say that a stream block can have "X numbers of textures of Y formats and Z size, and W bytes of vertex data". Artists can't sacrifice 4 textures to allow one texture to be 4 times the size, or sacrifice a texture to get more vertex data, or sacrifice some vertex data to get more texture space.
That is what you have to do if you don't trust the implementation's memory allocator to effectively recycle resources. For hard-load cases, it's generally OK to just delete all of the objects and expect the implementation to recycle effectively. But even here, it's a good idea to make sure that you don't try to allocate all of the available storage; leave the implementation a healthy percentage for fragmentation and so forth.
Of course, these kinds of things are a big part of the reason why IHVs have application-specific optimizations: they know up-front which resources were allocated for which purpose, and the driver makers can design allocation schemes that work with these assumptions.
Which is why Vulkan et. al exist: so that application developers don't have to rely on IHVs to work around problems in the API.