I think you are right. The general out-of-core describes exactly the situation in which we have limitted memory (RAM for CPU and global memory for GPU) compared to the task.
The term out-of-core typically refers to processing data that is too large to fit into a computer's main memory. Out-of-core - MachineLearningWTF
The term mentioned more, I believe, is out-of-core rendering, which is a broader concept that we don't have to rely on ray-tracing related approaches to do the rendering. However, the underlying problems to solve are the same:
- Geometry is too complex (tremendous amount of primitives)
- Textures are too detailed (thus, large)
- The scene is simply too large
So locally we usually use tiled-based rendering (and load / store tiles dynamically) and paging (may swap the finished part to the external storage, or directly output as files and do a final composition) or even some more complex pipelines for GPU (since host/device memory is quite different) to solve the lack of memory problem.
Note that I've seen no official definition from any literature before, but they do convey some very similar concepts for out-of-core ray-tracing (rendering):
 Kirill Garanzha, Alexander Bely, Simon Premoze, and Vladimir Galaktionov. 2011. Out-of-core GPU ray tracing of complex scenes. In ACM SIGGRAPH 2011 Talks (SIGGRAPH '11). Association for Computing Machinery, New York, NY, USA, Article 21, 1. https://doi.org/10.1145/2037826.2037854
 Budge, Brian, et al. "Out‐of‐core data management for path tracing on hybrid resources." Computer Graphics Forum. Vol. 28. No. 2. Oxford, UK: Blackwell Publishing Ltd, 2009.
 Rui Wang, Yuchi Huo, Yazhen Yuan, Kun Zhou, Wei Hua, and Hujun Bao. 2013. GPU-based out-of-core many-lights rendering. ACM Trans. Graph. 32, 6, Article 210 (November 2013), 10 pages. https://doi.org/10.1145/2508363.2508413
 Wald, Ingo, Andreas Dietrich, and Philipp Slusallek. "An interactive out-of-core rendering framework for visualizing massively complex models." ACM SIGGRAPH 2005 Courses. 2005. 17-es.
 Varadhan, Gokul, and Dinesh Manocha. "Out-of-core rendering of massive geometric environments." IEEE Visualization, 2002. VIS 2002.. IEEE, 2002.