lua - LSTM on top of CNN -

i have following lstm model implementation in torch, took here: https://github.com/wojzaremba/lstm/blob/master/main.lua

i have question following piece of code:

local function create_network()   local x                = nn.identity()()   local y                = nn.identity()()   local prev_s           = nn.identity()()   local                = {[0] = lookuptable(params.vocab_size,                                                     params.rnn_size)(x)}   local next_s           = {}   local split         = {prev_s:split(2 * params.layers)}   layer_idx = 1, params.layers     local prev_c         = split[2 * layer_idx - 1]     local prev_h         = split[2 * layer_idx]     local dropped        = nn.dropout(params.dropout)(i[layer_idx - 1])     local next_c, next_h = lstm(dropped, prev_c, prev_h)     table.insert(next_s, next_c)     table.insert(next_s, next_h)     i[layer_idx] = next_h   end   local h2y              = nn.linear(params.rnn_size, params.vocab_size)   local dropped          = nn.dropout(params.dropout)(i[params.layers])   local pred             = nn.logsoftmax()(h2y(dropped))   local err              = nn.classnllcriterion()({pred, y})   local module           = nn.gmodule({x, y, prev_s},                                       {err, nn.identity()(next_s)})   module:getparameters():uniform(-params.init_weight, params.init_weight)   return transfer_data(module) end 

in embedding part of lstm input, code uses lookuptable layer when handling ptb data base, i'm wondering how use lookuptable define other embedding different type of data. specially, input rgb images , embedding 1 of cnn models e.g alexnet without connected layers. ( https://gist.github.com/gcr/0bab9929dfee95164a4d )

its's looks fuzzy me. there better design purpose? how can create lstm on top of cnn model?

torch nn.lookuptable doing tensor index on weight tensor. in code specified used learn word vectors, since included in nngraph model. if have pre-trained model, can set it's weights lookuptable, time, shouldn't include in nngraph. dimensions of weight tensor shoud nindex (e.g. how many different images have) x noutput (e.g. lstm hidden size - rnn_size in code). or, don't have use lookuptable @ all, can specify input tensor directly.