Inside of my application I am trying to convert a MTLTexture into a CGImage or CIimage for use in other parts of the app. I am having a difficult time doing so because none of the example code I find seems to work properly.

For some reason something breaks on each and every one of them. Here are my attempts and why they dont work. If you could give me ideas on why that would be appreciated.

All of my textures are in brga8Unorm format.

Method 1 is to copy the pixels into an unsafe mutable pointer and use that to directly create a CGTexture. It creates a textures of all 0s

func toImage() -> NSImage? {
    let texture = self
    let width = texture.width
    let height = texture.height
    let bytesPerRow = width * 4
    let data = UnsafeMutableRawPointer.allocate(bytes: bytesPerRow * height, alignedTo: 4)
    defer {
        data.deallocate(bytes: bytesPerRow * height, alignedTo: 4)
    }

    let region = MTLRegionMake2D(0, 0, width, height)
    texture.getBytes(data, bytesPerRow: bytesPerRow, from: region, mipmapLevel: 0)
    var buffer = vImage_Buffer(data: data, height: UInt(height), width: UInt(width), rowBytes: bytesPerRow)

    let map: [UInt8] = [2, 1, 0, 3]
    vImagePermuteChannels_ARGB8888(&buffer, &buffer, map, 0)

    guard let colorSpace = CGColorSpace(name: CGColorSpace.genericRGBLinear) else { return nil }
    guard let context = CGContext(data: data, width: width, height: height, bitsPerComponent: 8, bytesPerRow: bytesPerRow,
            space: colorSpace, bitmapInfo: CGImageAlphaInfo.noneSkipLast.rawValue) else { return nil }
    guard let cgImage = context.makeImage() else { return nil }

    return NSImage(cgImage: cgImage, size: NSSize(width: width, height: height))
}

Method 2 tries to use CImage to do the job however CImage reports Cannot render image (with an input Metal texture) using a metal context. and returns nil

extension MTLTexture {
    func toImage() -> CGImage? {
        let context = CIContext()
        let texture = self
        let cImg = CIImage(mtlTexture: texture, options: nil)!
        let cgImg = context.createCGImage(cImg, from: cImg.extent)!
        return cgImg
    }
}

This last method doesnt even try to convert the image it is just trying to determine if the image is blank (all 0s) or not however even on fully green or even textured images it says they are blank.

func anythingHere(_ texture: MTLTexture) -> Bool {
    let width = texture.width
    let height = texture.height
    let bytesPerRow = width * 4

    let data = UnsafeMutableRawPointer.allocate(bytes: bytesPerRow * height, alignedTo: 4)
    defer {
        data.deallocate(bytes: bytesPerRow * height, alignedTo: 4)
    }

    let region = MTLRegionMake2D(0, 0, width, height)
    texture.getBytes(data, bytesPerRow: bytesPerRow, from: region, mipmapLevel: 0)
    var bind = data.assumingMemoryBound(to: UInt8.self)

    var sum:UInt8 = 0;
    for i in 0..<width*height {
        sum += bind.pointee
        bind = bind.advanced(by: 1)
    }
    return sum != 0
}

Clearly I am doing something fundamentally wrong as none of these methods are actually producing results. What am I doing wrong? What is the right way to do this?

  • Is it possible that the drawing has not completed yet? I've had this happen when I've forgotten to do a glFlush() in OpenGL or the equivalent in Metal (for example a completion handler on the command buffer). – user1118321 Mar 24 at 3:49

maybe you forget to call MTLBlitCommandEncoder.synchronizeTexture to sync VRAM to system memory.

please check: https://forums.developer.apple.com/thread/30488

and document: https://developer.apple.com/documentation/metal/mtlblitcommandencoder/1400757-synchronize

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.