A tensor is a (possibly multi-dimensional) array of numbers of a specified type with a defined size and shape. Due to their structured size and shape, operations on tensors can be optimized and parallelized using hardware accelerators, enhancing computational efficiency.