Kruskal Tensors

Copyright 2022 National Technology & Engineering Solutions of Sandia,
LLC (NTESS). Under the terms of Contract DE-NA0003525 with NTESS, the
U.S. Government retains certain rights in this software.

Kruskal format is a decomposition of a tensor \(\mathcal{X}\) as the sum of the outer products as the columns of matrices. For example, we might write:

\( \mathcal{X} = \sum_{r} a_r \circ b_r \circ c_r \)

where a subscript denotes column index and a circle denotes outer product. In other words, the tensor \(\mathcal{X}\) is built from the columns of the matrices \(A\), \(B\), and \(C\). It’s often helpful to explicitly specify a weight for each outer product, which we do here:

\( \mathcal{X} = \sum_{r} \lambda_r \, a_r \circ b_r \circ c_r \) The ktensor class stores the components of the tensor \(\mathcal{X}\) and can perform many operations, e.g., ttm, without explicitly forming the tensor \(\mathcal{X}\).

Kruskal tensor format via ktensor

Kruskal format stores a tensor as a sum of rank-1 outer products. For example, consider a tensor of the following form:

\( \mathcal{X} = a_1 \circ b_1 \circ c_1 + a_2 \circ b_2 \circ c_2 \)

This can be stored in Kruskal form as follows.

import pyttb as ttb
import numpy as np

np.random.seed(0)
A = np.random.rand(4, 2)  # First column is a_1, second is a_2.
B = np.random.rand(3, 2)  # Likewise for B.
C = np.random.rand(2, 2)  # Likewise for C.
X = ttb.ktensor([A, B, C])  # Create the ktensor.
X

ktensor of shape (4, 3, 2) with order F
weights=[1. 1.]
factor_matrices[0] =
[[0.5488135  0.71518937]
 [0.60276338 0.54488318]
 [0.4236548  0.64589411]
 [0.43758721 0.891773  ]]
factor_matrices[1] =
[[0.96366276 0.38344152]
 [0.79172504 0.52889492]
 [0.56804456 0.92559664]]
factor_matrices[2] =
[[0.07103606 0.0871293 ]
 [0.0202184  0.83261985]]

Y = ttb.ktensor(
    [np.random.rand(4, 1), np.random.rand(2, 1), np.random.rand(3, 1)]
)  # Another ktensor.
Y

ktensor of shape (4, 2, 3) with order F
weights=[1.]
factor_matrices[0] =
[[0.77815675]
 [0.87001215]
 [0.97861834]
 [0.79915856]]
factor_matrices[1] =
[[0.46147936]
 [0.78052918]]
factor_matrices[2] =
[[0.11827443]
 [0.63992102]
 [0.14335329]]

For Kruskal format, there can be any number of matrices, but every matrix must have the same number of columns. The number of rows can vary.

Specifying weights in a ktensor

Weights for each rank-1 tensor can be specified by passing in a column vector. For example:

\( \mathcal{X} = \lambda_1 \, a_1 \circ b_1 \circ c_1 + \lambda_2 \, a_2 \circ b_2 \circ c_2 \)

# Upcoming ktensors will be generated with this same initialization.
def generate_sample_ktensor() -> ttb.ktensor:
    np.random.seed(0)
    A = np.random.rand(4, 2)  # Create some data.
    B = np.random.rand(3, 2)
    C = np.random.rand(2, 2)
    weights = np.array([5.0, 0.25])

    return ttb.ktensor([A, B, C], weights)  # Create the ktensor.

X = generate_sample_ktensor()
X

ktensor of shape (4, 3, 2) with order F
weights=[5.   0.25]
factor_matrices[0] =
[[0.5488135  0.71518937]
 [0.60276338 0.54488318]
 [0.4236548  0.64589411]
 [0.43758721 0.891773  ]]
factor_matrices[1] =
[[0.96366276 0.38344152]
 [0.79172504 0.52889492]
 [0.56804456 0.92559664]]
factor_matrices[2] =
[[0.07103606 0.0871293 ]
 [0.0202184  0.83261985]]

Creating a one-dimensional ktensor

np.random.seed(0)
Y = ttb.ktensor([np.random.rand(4, 5)])  # A one-dimensional ktensor.
Y

ktensor of shape (4,) with order F
weights=[1. 1. 1. 1. 1.]
factor_matrices[0] =
[[0.5488135  0.71518937 0.60276338 0.54488318 0.4236548 ]
 [0.64589411 0.43758721 0.891773   0.96366276 0.38344152]
 [0.79172504 0.52889492 0.56804456 0.92559664 0.07103606]
 [0.0871293  0.0202184  0.83261985 0.77815675 0.87001215]]

Constituent parts of a ktensor

X = generate_sample_ktensor()
X.weights  # Weights or multipliers.

array([5.  , 0.25])

X.factor_matrices  # Cell array of matrices.

[array([[0.5488135 , 0.71518937],
        [0.60276338, 0.54488318],
        [0.4236548 , 0.64589411],
        [0.43758721, 0.891773  ]]),
 array([[0.96366276, 0.38344152],
        [0.79172504, 0.52889492],
        [0.56804456, 0.92559664]]),
 array([[0.07103606, 0.0871293 ],
        [0.0202184 , 0.83261985]])]

Creating a ktensor from its constituent parts

X = generate_sample_ktensor()
Y = ttb.ktensor(X.factor_matrices, X.weights)  # Recreate X.
Y

ktensor of shape (4, 3, 2) with order F
weights=[5.   0.25]
factor_matrices[0] =
[[0.5488135  0.71518937]
 [0.60276338 0.54488318]
 [0.4236548  0.64589411]
 [0.43758721 0.891773  ]]
factor_matrices[1] =
[[0.96366276 0.38344152]
 [0.79172504 0.52889492]
 [0.56804456 0.92559664]]
factor_matrices[2] =
[[0.07103606 0.0871293 ]
 [0.0202184  0.83261985]]

Creating an empty ktensor

Z = ttb.ktensor()  # Empty ktensor.
Z

ktensor of shape () with order F
weights=[]
factor_matrices=[]

Use full or to_tensor to convert a ktensor to a tensor

X = generate_sample_ktensor()
X.full()  # Converts to a tensor.

tensor of shape (4, 3, 2) with order F
data[:, :, 0] =
[[0.19381804 0.16256856 0.12514704]
 [0.21086124 0.17577751 0.13259822]
 [0.1504007  0.12657497 0.09849813]
 [0.15722304 0.13332549 0.10626643]]
data[:, :, 1] =
[[0.11054766 0.12266212 0.16930925]
 [0.10221034 0.10823074 0.13959484]
 [0.09282405 0.10501592 0.1487711 ]
 [0.11380622 0.13320036 0.19694404]]

X.to_tensor()  # Same as above.

tensor of shape (4, 3, 2) with order F
data[:, :, 0] =
[[0.19381804 0.16256856 0.12514704]
 [0.21086124 0.17577751 0.13259822]
 [0.1504007  0.12657497 0.09849813]
 [0.15722304 0.13332549 0.10626643]]
data[:, :, 1] =
[[0.11054766 0.12266212 0.16930925]
 [0.10221034 0.10823074 0.13959484]
 [0.09282405 0.10501592 0.1487711 ]
 [0.11380622 0.13320036 0.19694404]]

Use double to convert a ktensor to a multidimensional array

X = generate_sample_ktensor()
X.double()  # Converts to an array.

array([[[0.19381804, 0.11054766],
        [0.16256856, 0.12266212],
        [0.12514704, 0.16930925]],

       [[0.21086124, 0.10221034],
        [0.17577751, 0.10823074],
        [0.13259822, 0.13959484]],

       [[0.1504007 , 0.09282405],
        [0.12657497, 0.10501592],
        [0.09849813, 0.1487711 ]],

       [[0.15722304, 0.11380622],
        [0.13332549, 0.13320036],
        [0.10626643, 0.19694404]]])

Use tendiag or sptendiag to convert a ktensor to a ttensor

A ktensor can be regarded as a ttensor with a diagonal core.

X = generate_sample_ktensor()
R = len(X.weights)  # Number of factors in X.
core = ttb.tendiag(X.weights, ((R,) * X.ndims))  # Create a diagonal core.
Y = ttb.ttensor(core, X.factor_matrices)  # Assemble the ttensor
Y

Tensor of shape: (4, 3, 2)
	Core is a
		tensor of shape (2, 2, 2) with order F
		data[:, :, 0] =
		[[5. 0.]
		 [0. 0.]]
		data[:, :, 1] =
		[[0.   0.  ]
		 [0.   0.25]]
	U[0] = 
		[[0.5488135  0.71518937]
		 [0.60276338 0.54488318]
		 [0.4236548  0.64589411]
		 [0.43758721 0.891773  ]]
	U[1] = 
		[[0.96366276 0.38344152]
		 [0.79172504 0.52889492]
		 [0.56804456 0.92559664]]
	U[2] = 
		[[0.07103606 0.0871293 ]
		 [0.0202184  0.83261985]]

(X.full() - Y.full()).norm()  # They are the same.

7.601177430610147e-17

core = ttb.sptendiag(X.weights, ((R,) * X.ndims))  # Sparse diagonal core.
Y = ttb.ttensor(core, X.factor_matrices)  # Assemble the ttensor
Y

Tensor of shape: (4, 3, 2)
	Core is a
		sparse tensor of shape (2, 2, 2) with 2 nonzeros and order F
		[0, 0, 0] = 5.0
		[1, 1, 1] = 0.25
	U[0] = 
		[[0.5488135  0.71518937]
		 [0.60276338 0.54488318]
		 [0.4236548  0.64589411]
		 [0.43758721 0.891773  ]]
	U[1] = 
		[[0.96366276 0.38344152]
		 [0.79172504 0.52889492]
		 [0.56804456 0.92559664]]
	U[2] = 
		[[0.07103606 0.0871293 ]
		 [0.0202184  0.83261985]]

(X.full() - Y.full()).norm()  # They are the same.

7.601177430610147e-17

Use ndims and shape for the dimensions of a ktensor

X = generate_sample_ktensor()
X.ndims  # Number of dimensions.

3

X.shape  # Tuple of the shapes.

(4, 3, 2)

X.shape[1]  # Shape of the 2nd mode.

3

Subscripted reference for a ktensor

X = generate_sample_ktensor()
X.weights[1]  # Weight of the 2nd factor.

np.float64(0.25)

X.factor_matrices[1]  # Extract a matrix.

array([[0.96366276, 0.38344152],
       [0.79172504, 0.52889492],
       [0.56804456, 0.92559664]])

Subscripted assignment for a ktensor

X = generate_sample_ktensor()
X.weights = np.ones(X.weights.shape)  # Insert new multipliers.
X

ktensor of shape (4, 3, 2) with order F
weights=[1. 1.]
factor_matrices[0] =
[[0.5488135  0.71518937]
 [0.60276338 0.54488318]
 [0.4236548  0.64589411]
 [0.43758721 0.891773  ]]
factor_matrices[1] =
[[0.96366276 0.38344152]
 [0.79172504 0.52889492]
 [0.56804456 0.92559664]]
factor_matrices[2] =
[[0.07103606 0.0871293 ]
 [0.0202184  0.83261985]]

X.weights[0] = 7  # Change a single element of weights.

X.factor_matrices[2][:, [0]] = np.ones((2, 1))  # Change the matrix for mode 3.
X

ktensor of shape (4, 3, 2) with order F
weights=[7. 1.]
factor_matrices[0] =
[[0.5488135  0.71518937]
 [0.60276338 0.54488318]
 [0.4236548  0.64589411]
 [0.43758721 0.891773  ]]
factor_matrices[1] =
[[0.96366276 0.38344152]
 [0.79172504 0.52889492]
 [0.56804456 0.92559664]]
factor_matrices[2] =
[[1.         0.0871293 ]
 [1.         0.83261985]]

Using negative indexing for the last array index

X = generate_sample_ktensor()
X.factor_matrices[0][-1:, :]
X.shape

(4, 3, 2)

X = generate_sample_ktensor()
X.factor_matrices[0][0][
    1 : (np.prod(X.shape) - 1)
].item()  # Calculates factor_matrix[0][1]

0.7151893663724195

Adding and subtracting ktensors

Adding two ktensors is the same as concatenating the matrices.

np.random.seed(0)
X = ttb.ktensor(
    [np.random.rand(4, 2), np.random.rand(2, 2), np.random.rand(3, 2)]
)  # Data.
Y = ttb.ktensor(
    [np.random.rand(4, 2), np.random.rand(2, 2), np.random.rand(3, 2)]
)  # More data.
X, Y

(ktensor of shape (4, 2, 3) with order F
 weights=[1. 1.]
 factor_matrices[0] =
 [[0.5488135  0.71518937]
  [0.60276338 0.54488318]
  [0.4236548  0.64589411]
  [0.43758721 0.891773  ]]
 factor_matrices[1] =
 [[0.96366276 0.38344152]
  [0.79172504 0.52889492]]
 factor_matrices[2] =
 [[0.56804456 0.92559664]
  [0.07103606 0.0871293 ]
  [0.0202184  0.83261985]],
 ktensor of shape (4, 2, 3) with order F
 weights=[1. 1.]
 factor_matrices[0] =
 [[0.77815675 0.87001215]
  [0.97861834 0.79915856]
  [0.46147936 0.78052918]
  [0.11827443 0.63992102]]
 factor_matrices[1] =
 [[0.14335329 0.94466892]
  [0.52184832 0.41466194]]
 factor_matrices[2] =
 [[0.26455561 0.77423369]
  [0.45615033 0.56843395]
  [0.0187898  0.6176355 ]])

Z = X + Y  # Concatenates the factor matrices.
Z

ktensor of shape (4, 2, 3) with order F
weights=[1. 1. 1. 1.]
factor_matrices[0] =
[[0.5488135  0.71518937 0.77815675 0.87001215]
 [0.60276338 0.54488318 0.97861834 0.79915856]
 [0.4236548  0.64589411 0.46147936 0.78052918]
 [0.43758721 0.891773   0.11827443 0.63992102]]
factor_matrices[1] =
[[0.96366276 0.38344152 0.14335329 0.94466892]
 [0.79172504 0.52889492 0.52184832 0.41466194]]
factor_matrices[2] =
[[0.56804456 0.92559664 0.26455561 0.77423369]
 [0.07103606 0.0871293  0.45615033 0.56843395]
 [0.0202184  0.83261985 0.0187898  0.6176355 ]]

Z = X - Y  # Concatenates as with plus, but changes the weights.
Z

ktensor of shape (4, 2, 3) with order F
weights=[ 1.  1. -1. -1.]
factor_matrices[0] =
[[0.5488135  0.71518937 0.77815675 0.87001215]
 [0.60276338 0.54488318 0.97861834 0.79915856]
 [0.4236548  0.64589411 0.46147936 0.78052918]
 [0.43758721 0.891773   0.11827443 0.63992102]]
factor_matrices[1] =
[[0.96366276 0.38344152 0.14335329 0.94466892]
 [0.79172504 0.52889492 0.52184832 0.41466194]]
factor_matrices[2] =
[[0.56804456 0.92559664 0.26455561 0.77423369]
 [0.07103606 0.0871293  0.45615033 0.56843395]
 [0.0202184  0.83261985 0.0187898  0.6176355 ]]

(Z.full() - (X.full() - Y.full())).norm()  # Should be zero.

1.138063217752167e-16

Basic operations with a ktensor

+X  # Calls uplus.

ktensor of shape (4, 2, 3) with order F
weights=[1. 1.]
factor_matrices[0] =
[[0.5488135  0.71518937]
 [0.60276338 0.54488318]
 [0.4236548  0.64589411]
 [0.43758721 0.891773  ]]
factor_matrices[1] =
[[0.96366276 0.38344152]
 [0.79172504 0.52889492]]
factor_matrices[2] =
[[0.56804456 0.92559664]
 [0.07103606 0.0871293 ]
 [0.0202184  0.83261985]]

-X  # Calls uminus.

ktensor of shape (4, 2, 3) with order F
weights=[-1. -1.]
factor_matrices[0] =
[[0.5488135  0.71518937]
 [0.60276338 0.54488318]
 [0.4236548  0.64589411]
 [0.43758721 0.891773  ]]
factor_matrices[1] =
[[0.96366276 0.38344152]
 [0.79172504 0.52889492]]
factor_matrices[2] =
[[0.56804456 0.92559664]
 [0.07103606 0.0871293 ]
 [0.0202184  0.83261985]]

5 * X  # Calls mtimes.

ktensor of shape (4, 2, 3) with order F
weights=[5. 5.]
factor_matrices[0] =
[[0.5488135  0.71518937]
 [0.60276338 0.54488318]
 [0.4236548  0.64589411]
 [0.43758721 0.891773  ]]
factor_matrices[1] =
[[0.96366276 0.38344152]
 [0.79172504 0.52889492]]
factor_matrices[2] =
[[0.56804456 0.92559664]
 [0.07103606 0.0871293 ]
 [0.0202184  0.83261985]]

Use permute to reorder the modes of a ktensor

np.random.seed(0)
X = ttb.ktensor(
    [np.random.rand(4, 2), np.random.rand(2, 2), np.random.rand(3, 2)]
)  # Data.
X.permute(np.array((1, 2, 0)))  # Reorders modes of X.

ktensor of shape (2, 3, 4) with order F
weights=[1. 1.]
factor_matrices[0] =
[[0.96366276 0.38344152]
 [0.79172504 0.52889492]]
factor_matrices[1] =
[[0.56804456 0.92559664]
 [0.07103606 0.0871293 ]
 [0.0202184  0.83261985]]
factor_matrices[2] =
[[0.5488135  0.71518937]
 [0.60276338 0.54488318]
 [0.4236548  0.64589411]
 [0.43758721 0.891773  ]]

Use arrange to normalize the factors of a ktensor

The function arrange normalizes the columns of the factors and then arranges the rank-one pieces in decreasing order of shape.

np.random.seed(0)
X = ttb.ktensor(
    [np.random.rand(3, 2), np.random.rand(4, 2), np.random.rand(2, 2)]
)  # Unit weights.
X

ktensor of shape (3, 4, 2) with order F
weights=[1. 1.]
factor_matrices[0] =
[[0.5488135  0.71518937]
 [0.60276338 0.54488318]
 [0.4236548  0.64589411]]
factor_matrices[1] =
[[0.43758721 0.891773  ]
 [0.96366276 0.38344152]
 [0.79172504 0.52889492]
 [0.56804456 0.92559664]]
factor_matrices[2] =
[[0.07103606 0.0871293 ]
 [0.0202184  0.83261985]]

X.arrange()  # Normalized and rearranged.
X

ktensor of shape (3, 4, 2) with order F
weights=[1.33623212 0.09761402]
factor_matrices[0] =
[[0.64602825 0.59738279]
 [0.49219122 0.65610716]
 [0.58343407 0.4611477 ]]
factor_matrices[1] =
[[0.61851988 0.3041712 ]
 [0.26594907 0.66985152]
 [0.36683329 0.55033591]
 [0.64197943 0.3948534 ]]
factor_matrices[2] =
[[0.10407646 0.96180106]
 [0.9945693  0.27374937]]

Use fixsigns for sign indeterminacies in a ktensor

The largest magnitude entry for each factor is changed to be positive provided that we can flip the signs of pairs of vectors in that rank-1 component.

np.random.seed(0)
X = ttb.ktensor(
    [np.random.rand(4, 2), np.random.rand(2, 2), np.random.rand(3, 2)]
)  # Data.
Y = X
Y.factor_matrices[0][:, 0] = -Y.factor_matrices[0][
    :, 0
]  # switch the sign on a pair of columns
Y.factor_matrices[1][:, 0] = -Y.factor_matrices[1][:, 0]
Y

ktensor of shape (4, 2, 3) with order F
weights=[1. 1.]
factor_matrices[0] =
[[-0.5488135   0.71518937]
 [-0.60276338  0.54488318]
 [-0.4236548   0.64589411]
 [-0.43758721  0.891773  ]]
factor_matrices[1] =
[[-0.96366276  0.38344152]
 [-0.79172504  0.52889492]]
factor_matrices[2] =
[[0.56804456 0.92559664]
 [0.07103606 0.0871293 ]
 [0.0202184  0.83261985]]

Y.fixsigns()

ktensor of shape (4, 2, 3) with order F
weights=[1. 1.]
factor_matrices[0] =
[[0.5488135  0.71518937]
 [0.60276338 0.54488318]
 [0.4236548  0.64589411]
 [0.43758721 0.891773  ]]
factor_matrices[1] =
[[0.96366276 0.38344152]
 [0.79172504 0.52889492]]
factor_matrices[2] =
[[0.56804456 0.92559664]
 [0.07103606 0.0871293 ]
 [0.0202184  0.83261985]]

Use ktensor to store the ‘skinny’ SVD of a matrix

np.random.seed(0)
A = np.random.rand(4, 3)
A

array([[0.5488135 , 0.71518937, 0.60276338],
       [0.54488318, 0.4236548 , 0.64589411],
       [0.43758721, 0.891773  , 0.96366276],
       [0.38344152, 0.79172504, 0.52889492]])

[U, S, Vh] = np.linalg.svd(A, full_matrices=False)  # Compute the SVD.
# Numpy Expects U*S*Vh where pyttb expects U*S*V'
X = ttb.ktensor([U, Vh.transpose()], S)  # Store the SVD as a ktensor.
X

ktensor of shape (4, 3) with order F
weights=[2.21425506 0.29626672 0.21874633]
factor_matrices[0] =
[[-0.48665747 -0.08953811  0.54332309]
 [-0.41275168 -0.81483266  0.00407301]
 [-0.62010377  0.2250203  -0.73183668]
 [-0.45636813  0.52668447  0.41133746]]
factor_matrices[1] =
[[-0.42376582 -0.65045995  0.63033672]
 [-0.64907895  0.70346017  0.28955189]
 [-0.63175869 -0.2864361  -0.72030224]]

print(f"U*S*Vh:\n{U@np.diag(S)@Vh}")
print(
    f"\nX.factor_matrices[0]@np.diag(X.weights)@(X.factor_matrices[1].transpose()):\n\
{X.factor_matrices[0]@np.diag(X.weights)@(X.factor_matrices[1].transpose())}"
)
print(f"\nX.full():\n{X.full()}")  # Reassemble the original matrix.

U*S*Vh:
[[0.5488135  0.71518937 0.60276338]
 [0.54488318 0.4236548  0.64589411]
 [0.43758721 0.891773   0.96366276]
 [0.38344152 0.79172504 0.52889492]]

X.factor_matrices[0]@np.diag(X.weights)@(X.factor_matrices[1].transpose()):
[[0.5488135  0.71518937 0.60276338]
 [0.54488318 0.4236548  0.64589411]
 [0.43758721 0.891773   0.96366276]
 [0.38344152 0.79172504 0.52889492]]

X.full():
tensor of shape (4, 3) with order F
data[:, :] =
[[0.5488135  0.71518937 0.60276338]
 [0.54488318 0.4236548  0.64589411]
 [0.43758721 0.891773   0.96366276]
 [0.38344152 0.79172504 0.52889492]]

Displaying a ktensor

X = generate_sample_ktensor()
X

ktensor of shape (4, 3, 2) with order F
weights=[5.   0.25]
factor_matrices[0] =
[[0.5488135  0.71518937]
 [0.60276338 0.54488318]
 [0.4236548  0.64589411]
 [0.43758721 0.891773  ]]
factor_matrices[1] =
[[0.96366276 0.38344152]
 [0.79172504 0.52889492]
 [0.56804456 0.92559664]]
factor_matrices[2] =
[[0.07103606 0.0871293 ]
 [0.0202184  0.83261985]]

print(X)

ktensor of shape (4, 3, 2) with order F
weights=[5.   0.25]
factor_matrices[0] =
[[0.5488135  0.71518937]
 [0.60276338 0.54488318]
 [0.4236548  0.64589411]
 [0.43758721 0.891773  ]]
factor_matrices[1] =
[[0.96366276 0.38344152]
 [0.79172504 0.52889492]
 [0.56804456 0.92559664]]
factor_matrices[2] =
[[0.07103606 0.0871293 ]
 [0.0202184  0.83261985]]