It has been recently shown how the tensorial nature of real-time path integrals (PIs) involving the Feynman-Vernon influence functional can be utilized with matrix product states, taking advantage of the finite length of the bath-induced memory. Tensor networks (TNs) promise to provide a unified language to express the structure of a PI. Here, a generalized TN specifically incorporating the pairwise interaction structure of the influence functional and its invariance with respect to the average forward-backward position or the sojourn value in the form of the blip representation is derived and implemented. This pairwise connected TNPI (PC-TNPI) is illustrated through applications to typical spin-boson problems and explorations of the differences caused by the exact form of the spectral density. The storage and performance scalings are reported, showing the compactness of the representation and the efficiency of the contraction process. Finally, taking advantage of the compressed representation, the viability of using PC-TNPI for simulating multistate problems is demonstrated. The PC-TNPI structure can be shown to yield other TN algorithms currently in use. Consequently, it should be possible to use it as a starting point for deriving other optimized procedures.