High dimensional time series analysis has diverse applications in macroeconometrics and finance. Recent factor-type models employing tensor-based decompositions prove to be computationally involved due to the non-convex nature of the underlying optimization problem and also they do not capture the underlying temporal dependence of the latent factor structure. This work leverages the concept of tubal rank and develops a matrix-valued time series model, which first captures the temporal dependence in the data, and then the remainder signals across the time points are decomposed into two components: a low tubal rank tensor representing the baseline signals, and a sparse tensor capturing the additional idiosyncrasies in the signal. We address the issue of identifiability of various components in our model and subsequently develop a scalable Alternating Block Minimization algorithm to solve the convex regularized optimization problem for estimating the parameters. We provide finite sample error bounds under high dimensional scaling for the model parameters