Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CP Block Coordinate Descent #157

Open
wants to merge 26 commits into
base: master
Choose a base branch
from
Open

CP Block Coordinate Descent #157

wants to merge 26 commits into from

Conversation

kmp5VT
Copy link
Contributor

@kmp5VT kmp5VT commented May 18, 2023

A very interesting blocked version of the CP decomposition which under the hood uses ALS with fixed block sizes. Very similar to the paneled ALS method however, the difference comes from the following: After optimizing a block of size r<=R_{target} you compute the gradient of the target tensor. The following block of size r' now tries to approximate the gradient. This continues until there are no blocks left. if r = R, i.e. one block, one recovers the exact CP-ALS decomposition.

It is very interesting for two reasons:

  1. Using BCD allows one to fix the rank of the CP decomposition effectively eliminating it from the arithmetic intensity and replacing it with a prefactor of r * number of blocks.
  2. The overall decomposition accuracy is dependent on the blocksize. Choosing a blocksize too small will significantly reduce the accuracy of a decomposition. However, there seems to be a way to "saturate" the block-wise accuracy using a block-size r < R_{target}. This implies that some tensors (using tensors in general may be too strong assumption) are made up of a fixed number of component tensors T_{i,j,k} = A_{i,j,k} + B_{i,j,k} + C_{i,j,k} + \dots which themselves have relatively low CP rank. These component tensors are "discovered" as the gradient of T.

I have done limited testing with this method beyond getting it to work using a small DF coulomb integral tensors (water in the basis aug-cc-pVDZ/aug-cc-pVDZ-RI) and the random test tensors.

@kmp5VT kmp5VT requested a review from evaleev May 18, 2023 20:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant