Is there any memory efficient DenseNet implementation using Tensorflow

[ybsave]

So far, I cannot find any memory efficient implementation DenseNet implementation on Tensorflow. In the Torch codes, there are explicit assignment of shared memory. Would you please provide some hints on how to implement this on Tensorflow? Thank you.

[liuzhuang13]

Sorry I'm not quite sure how to do this in tensorflow.
@taineleau Can you help on this? Thanks!

[taineleau-zz]

Hi, please first check our technical report, which should give you enough knowledge to implement a memory-efficient DenseNet.
Basically, if you want to implement the memory-efficient version on a NN framework, this framework should allow you to assign the output of a specific operation manually (i.e., you can manually malloc the memory for the output). PyTorch and Caffe support this operation and MXNet partially support this. However, I am not familiar with Tensorflow so I am not sure how much workload it would be if it does not support the malloc things.

[ahundt]

Note I've got an outstanding feature request for the necessary operations on tensorflow itself at tensorflow/tensorflow#12948.

[taineleau-zz]

@ahundt good job! thanks for helping make a feature request on TF.

[joeyearsley]

https://github.com/joeyearsley/efficient_densenet_tensorflow

Made using gradient checkpointing like the gpleiss repo does.