Batch editing for Search & Replace #59

clozach · 2020-05-17T21:55:56Z

This PR constitutes a method for batch editing of Roam blocks and a handful of implemented scenarios

Features Demonstrated

Batch linking
Adding/Removing tags at the end of each block
General-purpose search-and-replace (incl. regex!)

Watch a quick demo of the first 2 bullet points here, and a demo pertinent to #35 here. 🤓

Implementation Notes

Making new variants of this workflow is super easy! Just pass a text manipulation function into withHighlightedBlocks.

To make this work, I've added 3 general-purpose functions:

…utils/dom.ts → getHighlightedBlocks() which returns parallel arrays of the currently-highlighted blocks and the content blocks nested therein
…utils/async.ts → forEachAsync() used to simulate async actions on each block
…roam/roam.ts → Roam.replace() which takes a string -> string function as an argument and uses it to flexibly update a node's contents

Known Issues

This PR is a minimum-viable-product implementation of search-and-replace. The ideal UI for this sort of thing would allow each replacement to be reviewed one step at a time, but that would require far more design work.
When implementing the add/remove tag features, I found myself wanting the highlighted nodes to "remain" highlighted after the change. As a result, withHighlightedBlocks does a naive "re-highlighting" after all of the blocks have been processed. Unfortunately, the highlight state appears to be maintained separately within Roam itself, resulting in blocks that remain highlighted even after hitting esc.
- Workarounds:
  - Edit a block, then hit esc to re-syncs the highlight state
  - If the block has a parent, collapsing the parent re-syncs the highlight state for the child
  - But probably you'll just want to reload the page

My hope is that someone smarter than me can find a better way to reestablish the highlights…perhaps by simulating keystrokes? If not, it may be worth deleting batch-editing.ts:106-113

# Batch Linking - Select one or more blocks (full blue highlight…not edit mode) - Press `Meta+shift+l` - Type a word or phrase to batch-convert to [[links]] - Press `Enter` Note: Batch Linking purposefully matches on whole-word boundaries only. This avoides accidentally linking words within other words, so batch-linking `ears` will turn `ears` into `[[ears]]`, but will leave `earshot` alone. This is in contrast to Roam's "Link All" feature, which will happily create a link to your page that looks like this: `[[ears]]hot`. # End Tags - Select one or more blocks (full blue highlight…not edit mode) - Press `Ctrl+shift+t` - Type a word to use as a tag - Press `Enter` The word you typed will be appended to every block, so… • One block • Another block …becomes… • One block #tagged • Another block #tagged # Known bugs - ✅ `tag` appends ` #tag`, but ❌ `urgent tag` appends ` #urgent tag`. - ❌ A number of roam-toolkit behaviors seem to break Undo, and these functions are no exception - ❌ While thse functions preserve the highlighted blocks, making it easy to toggle tags on-and-off, but the highlight is overly-permanent. Workaround: reload the page.

… batch operations

- tag → #tag - compund tag → #[[compound tag]] Note: batch tag removal works with this change out of the box

Stvad · 2020-05-18T01:38:00Z

Thanks! will review soon-ish

Stvad · 2020-05-18T05:52:22Z

First comment from just trying to use it - it should work for current active block too (i.e. if it's in edit mode)

Stvad · 2020-05-18T05:54:16Z

Also the highlighting bug is confusing. If we can't fix it I'd prefer to loose the highlighting rather then to have it

Stvad · 2020-05-18T05:56:30Z

I'm surprised that this breaks undo, other toolkit functions don't seem to.

Stvad · 2020-05-18T05:57:05Z

btw can you elaborate on use-case for add/remove tag at the end?

src/ts/features/batch-editing.ts

Stvad · 2020-05-19T02:59:13Z

I think looking at #63 would be useful for figuring out the highlighting stuff

Stvad · 2020-05-19T02:45:48Z

src/ts/features/batch-editing.ts

+        const regex = new RegExp(`\\[\\[${text}]]|(\\b${text}\\b)`, 'g')
+        return originalString.replace(regex, function (m, group1) {
+            if (!group1) return m
+            else return `[[${m}]]`
+        })


I think this can be made simpler by using regex like [^\\[]\\b${text}\\b[^\\]] and then just replacing all matches

I tried this…

const regex = new RegExp(`[^\\[]\\b${text}\\b[^\\]]`, 'g') return originalString.replace(regex, `[[${text}]]`)

…and it doesn't work:

Create a Bracket Link.

becomes

Create a[[Bracket Link]]

¯_(ツ)_/¯

Stvad · 2020-05-19T02:46:51Z

src/ts/features/batch-editing.ts

+    if (!text || text === '') return
+
+    withHighlightedBlocks(originalString => {
+        if (text.includes(' ')) {


I think there are a bunch of other things just # does not work with. I think it'd be simpler to just always add brackets

Stvad · 2020-05-19T02:48:14Z

src/ts/features/batch-editing.ts

+        return
+
+    withHighlightedBlocks(originalString => {
+        const regex = new RegExp(`(.*) (#.*)`)


this does not seem like it would handle newlines?

also if you don't need to use template string/etc the /(.*) (#.*)/ syntax is a bit nicer

I think just doing something like "find last # " and take a substring before that as a result may be simpler here vs using regex

Good potential catch. I tested it out and it actually seems to work fine with newlines, assuming you mean the sort that allow you to add whitespace within a single block (Shift-return on my Mac).

Good point re: template string. I've replaced it with an inlined regex literal. :)

Stvad · 2020-05-19T03:04:01Z

src/ts/features/batch-editing.ts

+const withHighlightedBlocks = (mod: { (orig: string): string }) => {
+    const highlighted = getHighlightedBlocks()
+
+    const contentBlocks = Array.from(highlighted.contentBlocks)


shoul getHighlightedBlocks return array straight away?

Stvad · 2020-05-19T03:05:53Z

src/ts/roam/roam.ts

+        if (!textarea) {
+            console.log("🚨 NO TEXTAREA returned from ", element)
+            return
+        }
+
+        const newText = mod(textarea.value)
+
+        Roam.save(new RoamNode(newText))


There is the apply function that does basically this

Stvad · 2020-05-19T03:06:00Z

src/ts/roam/roam.ts

+        Keyboard.pressEsc()
+        Keyboard.pressEsc()


also, I'm not sure if this should be responsibility of this function

Stvad · 2020-05-19T03:09:16Z

src/ts/roam/roam.ts

@@ -77,6 +84,20 @@ export const Roam = {
        this.applyToCurrent(node => node.withCursorAtTheEnd())
    },

+    async replace(element: HTMLElement, mod: { (orig: string): string }) {


so this is basically apply but for HtmlElement instead of current block, I feel like they should follow similar interface (i.e. in terms of RoamNode)

Stvad · 2020-05-19T03:10:06Z

src/ts/roam/roam.ts

+            // resulting block before Roam has had a chance to
+            // replace the <span> with a <textarea>. Without this,
+            // Batch operations often skip some blocks.
+            await delay(100)


leftClick has additional delay parameter

also is 100 an absolute minimum we can live with?

Stvad · 2020-05-19T03:14:09Z

src/ts/utils/dom.ts

@@ -33,6 +33,18 @@ export function getFirstTopLevelBlock() {
    return firstChild.querySelector('.roam-block, textarea') as HTMLElement
 }

+export function getHighlightedBlocks(): { parentBlocks: NodeListOf<HTMLElement>, contentBlocks: NodeListOf<HTMLElement> } {


should probably be 2 separate functions

README.md

clozach added 5 commits May 17, 2020 14:27

Updated README

477129e

Fix race condition that sometimes causes blocks to get skipped during…

f89a500

… batch operations

Add support for multi-word tags when batch-tagging

0b2fbfb

- tag → #tag - compund tag → #[[compound tag]] Note: batch tag removal works with this change out of the box

Add support for multi-block search-and-replace, includnig regex!

c51faa6

Stvad reviewed May 18, 2020

View reviewed changes

src/ts/features/batch-editing.ts Show resolved Hide resolved

src/ts/features/batch-editing.ts Show resolved Hide resolved

src/ts/features/batch-editing.ts Show resolved Hide resolved

Stvad reviewed May 19, 2020

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Batch editing for Search & Replace #59

Batch editing for Search & Replace #59

clozach commented May 17, 2020

Stvad commented May 18, 2020

Stvad commented May 18, 2020

Stvad commented May 18, 2020

Stvad commented May 18, 2020

Stvad commented May 18, 2020

Stvad commented May 19, 2020

Stvad May 19, 2020

clozach May 19, 2020

Stvad May 19, 2020

Stvad May 19, 2020

Stvad May 19, 2020

Stvad May 19, 2020

clozach May 19, 2020

Stvad May 19, 2020

Stvad May 19, 2020

Stvad May 19, 2020

Stvad May 19, 2020

Stvad May 19, 2020

Stvad May 19, 2020

Stvad May 19, 2020

Stvad May 19, 2020

Batch editing for Search & Replace #59

Are you sure you want to change the base?

Batch editing for Search & Replace #59

Conversation

clozach commented May 17, 2020

Features Demonstrated

Implementation Notes

Known Issues

Stvad commented May 18, 2020

Stvad commented May 18, 2020

Stvad commented May 18, 2020

Stvad commented May 18, 2020

Stvad commented May 18, 2020

Stvad commented May 19, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment