test: Restrict types inside the save function #1264
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We need help from the compiler so these interfaces will be useful for determining the shape of the data.
The interfaces should match the actual data types that get passed in.
product-auto-label
bot
added
size: s
These properties are actually optional
product-auto-label
bot
added
size: m
save doesn’t encode strings or ints or any values that don’t have keys properly so we should restrict the interface to what it does include.
danieljbruce
changed the title
daniel-sanche
approved these changes
Aug 26, 2024
src/interfaces/save.ts
Outdated
| Entity data passed into save in an array form will be of type SaveArrayData | ||
| so will have name and value defined because they are needed in these places: | ||
| https://github.com/googleapis/nodejs-datastore/blob/bf3dafd8267c447a52f7764505042a60b1a9fd28/src/index.ts#L1152 | ||
| https://github.com/googleapis/nodejs-datastore/blob/bf3dafd8267c447a52f7764505042a60b1a9fd28/src/index.ts#L1134 |
There was a problem hiding this comment.
These links seem likely to become irrelevant over time. Can the prurpose of this interface be described in a more general way?
There was a problem hiding this comment.
Yeah. I just updated it. The links are permalinks, but I did just update the description inline.
| An Entity passed into save will include a Key object contained either inside | ||
| a `key` property or inside a property indexed by the Key Symbol. If it is | ||
| the former then it will be of type SaveEntityWithoutKeySymbol. | ||
| */ |
There was a problem hiding this comment.
This part is a little unclear to me, I don't understand what the Key Symbol is or how it's used. Maybe that's just because I'm not familiar with the repo though
There was a problem hiding this comment.
nodejs-datastore/src/entity.ts
Line 49 in bf3dafd
danieljbruce
added a commit
that referenced
this pull request
Aug 28, 2024
… of nested fields (#1266) * test: Restrict types inside the save function (#1264) * Add interfaces that restrict values passed to save * Add interfaces for save We need help from the compiler so these interfaces will be useful for determining the shape of the data. * Modify current interfaces to make a closer match The interfaces should match the actual data types that get passed in. * key and data in PrepareEntityObjectResponse option These properties are actually optional * Document all the interfaces added for the save fn * Add two tests showing name/value should be provide * Make the SaveNonArrayData more specific. save doesn’t encode strings or ints or any values that don’t have keys properly so we should restrict the interface to what it does include. * Add header * Test should be more flexible * Add a class for elements with ToString * Define the toString class inline * Eliminate links to points in code * test: Add tests that describe how save should encode excludeFromIndexes for a complex case (#1263) * Write a sample test for what save output should be * Add a test for a bunch of complex excludeFromIndex * Change the replace function so it works everywhere * Modify the expected value Make it contain longStringArray * Correct the test for evaluating arrays * Break up the entityToEntityProto into separate par * Fix the tests to work with new addExcludeFromIndex * Revert "Break up the entityToEntityProto into separate par" This reverts commit 60dabd7. * Revert "Fix the tests to work with new addExcludeFromIndex" This reverts commit ba4e82b. * Remove source code changes * Skip the test that looks at arrays * Remove some of the wildcard indexes This is in excludeFromIndexes - replace them with literals. * Remove another boolean value from excludeFromIndex * Eliminate duplicate expectedMutations * Eliminate duplicate runTest function * Begin to set up the async tests * Pack all tests into the async framework * Delete tests addressed by async * Get rid of test functions and inline everything * Add comment * Add comments describing each test case * Remove only * Add another test looks at name/value not in array * Add a test for excludeLarge properties and name/va * Fix the test so that the array case matches Matches the single case * should pass the right properties for an array * Rename the test * Change name to entityName * Add two tests that capture the array encoding encoding problem * Change expected output of arrays * Remove only * Add 2 more tests to ensure behaviour is preserved * Correct test - it should apply excludeFromIndexes * Update the test for the nested value * fix: Handle excludeLarge properties for arrays properly (#1265) * Write a sample test for what save output should be * Add a test for a bunch of complex excludeFromIndex * Change the replace function so it works everywhere * Modify the expected value Make it contain longStringArray * Correct the test for evaluating arrays * Break up the entityToEntityProto into separate par * Fix the tests to work with new addExcludeFromIndex * Break up the entityToEntityProto into separate par * Fix the tests to work with new addExcludeFromIndex * Revert "Break up the entityToEntityProto into separate par" This reverts commit 60dabd7. * Revert "Fix the tests to work with new addExcludeFromIndex" This reverts commit ba4e82b. * Remove source code changes * Skip the test that looks at arrays * Remove some of the wildcard indexes This is in excludeFromIndexes - replace them with literals. * Remove another boolean value from excludeFromIndex * Eliminate duplicate expectedMutations * Eliminate duplicate runTest function * Begin to set up the async tests * Pack all tests into the async framework * Delete tests addressed by async * Get rid of test functions and inline everything * Add comment * Add comments describing each test case * Remove only * Add another test looks at name/value not in array * Add a test for excludeLarge properties and name/va * Fix the test so that the array case matches Matches the single case * should pass the right properties for an array * Rename the test * Change name to entityName * Add two tests that capture the array encoding encoding problem * Change expected output of arrays * Remove only * Add 2 more tests to ensure behaviour is preserved * Make argument more specific for excludeFromIndexes * Use excludeLargeProperties for array/non-array cas * Fix the test to include entity proto * Correct mistakes in initial implementation * Don’t skip any of the tests * Remove only * Add check for excludeLargeProperties * Remove TODO * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
To solve the problem described in go/excludefromindexes-nested-fields it is important to explore and document what kind of data can be passed into the save function. This PR defines interfaces and further restricts non-user facing variable types in the save source code so that the compiler will tell us when the source code is wrong. Right now, the first argument of
saveis of typeEntitieswhich is an alias forany.Note: This PR doesn't change how the source code functions at all.
Changes
SaveArrayDatainterface is defined so that the compiler forces array data to match a type internally that won't throw an error.SaveNonArrayDatais defined for valid input data that is not provided in array formSaveArrayDataandSaveNonArrayDatainterfaces acting as building blocks, a hierarchy of other interfaces are defined that eventually define theSaveEntityinterface which is now used in the save function to better restrict the data types that can be used within that function.google.datastore.v1.Keyandgoogle.datastore.v1.Entityare references to functions andkeyanddataproperties do not store functions.Next steps
It would be useful to further restrict the types within the save function to make it easier to work with the complex data structures that are involved with the problem from go/excludefromindexes-nested-fields