Скачать или смотреть How to Create a Tensor View in PyTorch Without Data Copying

Learn about creating tensor views in PyTorch without data copying. This guide explains the limitations and alternative solutions for manipulating tensors effectively.
---
This video is based on the question https://stackoverflow.com/q/62921102/ asked by the user 'Dohn Joe' ( https://stackoverflow.com/u/9545531/ ) and on the answer https://stackoverflow.com/a/62950668/ provided by the user 'Marius' ( https://stackoverflow.com/u/4182746/ ) at 'Stack Overflow' website. Thanks to these great users and Stackexchange community for their contributions.

Visit these links for original content and any more details, such as alternate solutions, latest updates/developments on topic, comments, revision history etc. For example, the original title of the Question was: Create a tensor view (no data copy) where one row/col is removed

Also, Content (except music) licensed under CC BY-SA https://meta.stackexchange.com/help/l...
The original Question post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license, and the original Answer post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license.

If anything seems off to you, please feel free to write me at vlogize [AT] gmail [DOT] com.
---
How to Create a Tensor View in PyTorch Without Data Copying

Introduction

In the world of machine learning and deep learning with PyTorch, memory management is crucial. Particularly, if you're working with large datasets, you may often find yourself needing to manipulate tensors efficiently. A common requirement is to delete or remove rows or columns from a tensor while ensuring no copies of the data are made. This question becomes pivotal when dealing with a large tensor of shape (2000, 10000), and wanting to create tensor views that facilitate model training.

The Problem at Hand

The situation arises when:

You have a large tensor of shape (2000, 10000).

You want to remove a specific row or column (for instance, a specific feature) without creating a memory-consuming data copy.

The objective is to prepare tuples of new views for feature training (e.g., one target feature alongside inputs from the remaining features) to streamline training with multiple models simultaneously.

Unfortunately, as of the current state of PyTorch, accomplishing this goal isn't straightforward.

Understanding Tensor View Capabilities in PyTorch

Limitations of Current Solutions

Dense Tensors:

In PyTorch, dense tensors are represented by a contiguous data array. When you create a view of the tensor, the shape changes by simply altering the indices rather than removing any data outright.

Sparse Tensors:

Sparse tensors, on the other hand, do not provide the functionality needed in this case. They generally spread out densely stored data instead of allowing for selective removals.

Alternatives to Deleting Rows/Columns

Since directly creating a tensor view that omits a row or column without copying data isn't achievable, we have to consider alternative approaches that still maintain efficiency.

Adjusting Model Design

One viable solution is to adapt your model to work with the full tensor and internally manage the exclusion of unwanted features. Here’s how you can do it:

Using Linear Layers: If your first layer is torch.nn.Linear, you can compute the outputs by simply adjusting the output to exclude the effect of the target feature:

[[See Video to Reveal this Text or Code Snippet]]

This essentially subtracts the contribution of the target feature from the output.

Weight Adjustment for Final Models: For the final model representation, you can modify the weights of your linear layer to reflect the absence of the removed feature.

Training Efficiency Considerations

Single vs. Multiple Models: Instead of training 10k individual models (one for each feature), consider designing a single model with 10k outputs. Each output can depend on a mix of inputs while ensuring that each feature only processes 9999 of the input features.

This approach reduces operational overhead and increases efficiency while maintaining manageable complexity.

Conclusion

While PyTorch currently does not support efficient tensor viewing with row or column deletions without data copying, there are still effective strategies to manage this problem. By adjusting model architectures and thinking creatively about how to handle inputs, you can optimize memory use and enhance computational efficiency.

If you find yourself facing this challenge, consider the internal adjustments and a streamlined approach to training multiple outputs simultaneously. Embracing a unified model design may just be the key to handling the large datasets you've been working with more efficiently.