Merge branch 'refs/heads/main' into sync-prod

.asf.yaml

@@ -16,7 +16,7 @@ github:
     production
 
 notifications:
-  jobs:         janhoy@apache.org
-  issues:       janhoy@apache.org
-  pullrequests: janhoy@apache.org
+  jobs:         builds@solr.apache.org
+  issues:       issues@solr.apache.org
+  pullrequests: issues@solr.apache.org
   jira_options: link label worklog

.github/workflows/pr-build-pelican.yml

@@ -10,25 +10,10 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - uses: actions/checkout@v4
-      - name: Set up Python
-        # This is the version of the action for setting up Python, not the Python version.
-        uses: actions/setup-python@v5
+      - name: Build Pelican Site
+        uses: apache/infrastructure-actions/pelican@main
         with:
-          python-version: '3.x'
-      - name: Display Python version
-        run: python -c "import sys; print(sys.version)"
-      - name: Install Pelican
-        shell: bash
-        # Install needs to run in separate shell so stdout is restored
-        run: |
-          (
-            pip3 install pelican==4.9.1 markdown bs4 ezt requests 
-          )
-          python3 -V
-          echo "Pelican version:"
-          pelican --version
-      # If the site uses Github Flavored Markdown, use this build branch
-      - name: Generate website from markdown
-        shell: bash
-        run: |
-          python3 -B -m pelican content
+          gfm: 'false'
+          version: '4.9.1'
+          publish: 'false'
+          debug: 'false'

README.md

@@ -4,54 +4,31 @@ This repository contains the "source code" of the Solr website at [solr.apache.o
 
 ## Building the site
 
-The site is written in [Markdown][9] syntax and built into a static site using [Pelican][1]. The site is re-built automatically by [ASF Buildbot][5] on every push to main branch, and the result can be previewed at [solr.staged.apache.org][6]. Build success/failure emails are sent to [commits@solr.apache.org][7] mailing list. Read more about the mechanics behind auto building in [INFRA Confluence][8].
+The site is written in [Markdown][9] syntax and built into a static site using [Pelican][1].
+
+On each Pull Request we do a simple pelican build. The staging site is re-built automatically by Github Actions on every push to `main` branch, and the result can be previewed at [solr.staged.apache.org][6]. Build success/failure emails are sent to [commits@solr.apache.org][7] mailing list.
 
 If the staged site looks good, simply merge the changes to branch `production` and the site will be deployed in a minute or two. Note that simple edits can also be done directly in the GitHub UI rather than clone -> edit -> commit -> push.
 
 > **IMPORTANT**: Please never commit directly to `production` branch. All commits should go to `main, and then merge `main` to `production`. Note that it **is** possible to make a Pull Request for the merge from `main-->production`. If you do so, please merge using a merge commit rather than a squash merge.
 
-For larger edits it is recommended to build and preview the site locally. This lets you see the result of your changes instantly without committing anything. The next sections detail that procedure. The TL;DR instructions goes like this:
+For larger edits it is recommended to build and preview the site locally. This lets you see the result of your changes instantly without committing anything.
+The bundled script uses a docker image to build and serve the site locally. Please make sure you have docker installed.
 
-    # Usage: ./build.sh [-l] [<other pelican arguments>]
+    # Usage: ./build.sh [-l] [-h] [<other pelican arguments>]
     #        -l     Live build and reload source changes on localhost:8000
     #        --help Show full help for options that Pelican accepts
     ./build.sh -l
 
 Now go to <http://localhost:8000> to view the beautiful Solr web page served from your laptop with live-preview of updates :)
 
-### Installing Pelican by hand
-
-The site uses [Pelican][1] for static html generation. Pelican requires [Python 3.5+][4] and can be installed with pip.
-
-**The `build.sh` script mentioned in the above paragraph takes care of setting up your Pelican environment,** and you can skip this part unless you want to understand the moving parts and install things by hand. Assuming that you have python3 installed, simply run:
-
-```sh
-pip3 install -r requirements.txt
-```
-
-If you run into conflicts with existing packages, a solution is to use a virtual Python environment. See the [Pelican installation page][2] for more details. These are quick commands, Linux flavor:
-
-```sh
-python3 -m venv env
-source env/bin/activate
-pip install -r requirements.txt
-```
-
-Once Pelican is installed you can convert your content into HTML via the pelican command (`content` is the default location to build from).
-
-```sh
-pelican
-```
-
-The above command will generate your site and save it in the `output/` folder using the solr theme and settings defined in `pelicanconf.py`
+### Other options
 
-You can also tell Pelican to watch for your modifications, instead of manually re-running it every time you want to see your changes. To enable this, run the pelican command with the `-r` or `--autoreload` option. On non-Windows environments, this option can also be combined with the `-l` or `--listen` option to simultaneously both auto-regenerate and serve the output through a builtin webserver on <http://localhost:8000>.
+If you want to build the site without the docker image, you can install Python 3 and Pelican, see [manual install](./manual-install.md) for details.
 
-```sh
-pelican --autoreload --listen
-```
+On Windows, you can use the Windows Subsystem for Linux (WSL) to run the build script. Or you can run the docker command directly in a Terminal:
 
-Remember that on Mac/Linux you can use the `build.sh` script with `-l` option to do the same.
+    docker run --rm -ti -w /work -p 8000:8000 -v $(pwd):/work python:3-alpine sh -c "pip3 install -r requirements.txt; pelican content -r -l -b 0.0.0.0"
 
 ## Updating site during a Solr release
 
@@ -80,9 +57,7 @@ Modify `SOLR_LATEST_RELEASE` and `SOLR_PREVIOUS_MAJOR_RELEASE`, and
 [2]: https://docs.getpelican.com/en/stable/install.html
 [3]: https://solr.apache.org/downloads.html#about-versions-and-support
 [4]: https://www.python.org/downloads/
-[5]: https://ci2.apache.org/#/builders/3
 [6]: https://solr.staged.apache.org
 [7]: https://lists.apache.org/list.html?commits@solr.apache.org
-[8]: https://wiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features
 [9]: http://daringfireball.net/projects/markdown/syntax
 

build.sh

@@ -14,51 +14,122 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-if [[ ! -z $1 ]]; then
-  if [[ "$1" == "-l" ]]; then
-    SERVE=true
-    shift
+# Fail on error
+set -e
+#set -x
+
+PYTHON_IMAGE="python:3-alpine"
+SOLR_LOCAL_PELICAN_IMAGE="solr-pelican-image"
+DOCKER_CMD="docker run --rm -ti -w /work -p 8000:8000 -v $(pwd):/work $SOLR_LOCAL_PELICAN_IMAGE"
+unset SERVE
+PIP_CMD="pip3 install -r requirements.txt"
+PELICAN_CMD="pelican content -o output"
+PELICAN_OPTS=""
+export SITEURL="https://solr.apache.org/"
+
+function usage {
+   echo "Usage: ./build.sh [-l] [-h] [<other pelican arguments>]"
+   echo "       -l     Live build and reload source changes on localhost:8000"
+   echo "       --help Show full help for options that Pelican accepts"
+}
+
+function build_image {
+  echo "Building local Docker image for Pelican, called $SOLR_LOCAL_PELICAN_IMAGE."
+  # Make a new local image with the pip packages installed
+  docker rm -f solr-pelican >/dev/null 2>&1 || true
+  docker run --name solr-pelican -w /work -v $(pwd):/work $PYTHON_IMAGE sh -c "$PIP_CMD"
+  docker commit solr-pelican $SOLR_LOCAL_PELICAN_IMAGE
+  docker rm -f solr-pelican >/dev/null 2>&1 || true
+}
+
+function ensure_image {
+  if ! docker inspect $SOLR_LOCAL_PELICAN_IMAGE >/dev/null 2>&1
+  then
+    build_image
+  fi
+}
+
+function check_requirements_update {
+  # Get the last modified time of requirements.txt
+  local req_mod_time
+  if [[ $(uname) == "Darwin" ]]; then
+    req_mod_time=$(stat -f "%m" requirements.txt)
   else
-    echo "Usage: ./build.sh [-l] [<other pelican arguments>]"
-    echo "       -l     Live build and reload source changes on localhost:8000"
-    echo "       --help Show full help for options that Pelican accepts"
-    if [[ "$1" == "-h" ]]; then
-      exit 0
-    elif [[ "$1" == "--help" ]]; then
-      echo
-      echo "Below is a list of other arguments you can use which will be passed to pelican."
-      echo
-      pelican --help
-      exit 0
-    fi
+    req_mod_time=$(stat -c "%Y" requirements.txt)
   fi
-fi
-if [[ ! $(python3 -h 2>/dev/null) ]]; then
-  echo "No python installed"
-  echo "Try one of"
-  echo "  brew install python3"
-  echo "  apt install python3"
-  exit 2
-fi
-if [[ -d env ]]; then
-  source env/bin/activate && pip -q install --upgrade pip && pip -q install -r requirements.txt >/dev/null
-fi
-if [[ ! $(pelican -h 2>/dev/null) ]]; then
-  echo "No pelican installed, attempting install"
-  python3 -m venv env && source env/bin/activate && pip -q install --upgrade pip && pip install -r requirements.txt
-  if [[ $? -gt 0 ]]; then
-    echo "Failed pelican install, exiting."
-    exit 2
+
+  # Get the build timestamp of the docker image
+  local image_build_time
+  image_build_time=$(docker inspect --format='{{.Created}}' $SOLR_LOCAL_PELICAN_IMAGE)
+
+  # Parse the timestamp into seconds since epoch in UTC
+  if [[ $(uname) == "Darwin" ]]; then
+    # macOS date command workaround
+    image_build_time=$(echo "$image_build_time" | awk -F '.' '{print $1}')
+    image_build_time=$(date -ju -f "%Y-%m-%dT%H:%M:%S" "$image_build_time" "+%s")
   else
-    echo "Install OK" && echo && echo
+    # Linux date command
+    image_build_time=$(date -d "$(echo "$image_build_time" | cut -d'.' -f1 | sed 's/T/ /; s/Z//')" --utc "+%s")
+  fi
+
+  # Compare the timestamps and build the image if requirements.txt is newer
+  if [[ $req_mod_time -gt $image_build_time ]]; then
+    echo "requirements.txt has been updated since the last build, rebuilding image!"
+    build_image
   fi
+}
+
+if ! docker -v >/dev/null 2>&1
+then
+  echo "ERROR: This script requires docker."
+  echo "       Please install Docker and try again."
+  echo
+  usage
+  exit 2
 fi
+
+while getopts ":lbh-:" opt; do
+  case ${opt} in
+    l )
+      SERVE=true
+      ;;
+    b )
+      build_image
+      ;;
+    h )
+      usage
+      exit 0
+      ;;
+    - )
+      case "${OPTARG}" in
+        help )
+          usage
+          echo
+          echo "Below is a list of other arguments you can use which will be passed to pelican."
+          echo
+          $DOCKER_CMD pelican -h
+          exit 0
+          ;;
+        * )
+          PELICAN_OPTS+="--${OPTARG} "
+          ;;
+      esac
+      ;;
+    \? )
+      PELICAN_OPTS+="-${OPTARG} "
+      ;;
+  esac
+done
+shift $((OPTIND -1))
+
+ensure_image
+check_requirements_update
 if [[ $SERVE ]]; then
   echo "Building Solr site locally. Goto http://localhost:8000 to view."
   echo "Edits you do to the source tree will be compiled immediately!"
-  pelican --autoreload --listen $@
+  $DOCKER_CMD sh -c "$PELICAN_CMD --autoreload --listen -b 0.0.0.0 $PELICAN_OPTS $*"
 else
-  echo "Building Solr site."
+  echo "Building Solr site locally."
   echo "To build and serve live edits locally, run this script with -l argument. Use -h for help."
-  pelican $@
+  $DOCKER_CMD sh -c "$PELICAN_CMD $PELICAN_OPTS $*"
 fi

content/pages/blogposts/2024-07-01-birds-of-a-feather.md

@@ -0,0 +1,11 @@
+Title: Community Over Code EU 2024 Apache Lucene/Solr Birds Of A Feather
+category: solr/blogposts
+summary: This blog post is a summary of the Apache Lucene/Solr Birds of a Feather from Community Over Code EU 2024 in Bratislava, Slovakia. Written by Alessandro Benedetti, Director at Sease and Apache Solr Committer and PMC member.
+slug: community-over-code-eu-2024-birds-of-a-feather
+URL: blogposts/community-over-code-eu-2024-birds-of-a-feather.html
+save_as: blogposts/community-over-code-eu-2024-birds-of-a-feather.html
+
+# Community Over Code EU 2024 Apache Lucene/Solr Birds Of A Feather
+Dive into the insights and key takeaways from the Apache Lucene/Solr Birds of a Feather (BoF) session at Community Over Code EU 2024, held in the vibrant city of Bratislava, Slovakia. This session was specially organized to gather feedback from the community, fostering a collaborative environment where developers, users, and enthusiasts could share their experiences and insights. 
+
+Read about the brainstorming that took place during the event to shape the future of these Apache projects - the complete post can be read here: [Community Over Code EU 2024 Apache Lucene/Solr Birds Of A Feather](https://sease.io/2024/06/community-over-code-eu-2024.html)

manual-install.md

@@ -0,0 +1,49 @@
+# Installing Pelican by hand
+
+The site uses [Pelican][1] for static html generation. Pelican requires [Python 3.5+][2] and can be installed with pip.
+
+**The `build.sh` script mentioned in README is the easiest way of building the site using Docker**.
+If for some reason you want to install Python and Pelican by hand, here are the steps: 
+
+## Install Python 3
+
+First, you need to install Python 3. You can download the latest version from the [Python website][2] or
+use your package manager to install it. For example, on macOS:
+
+```shell
+brew install python
+```
+
+## Install Pelican
+
+To install pelican and requirements, simply run the following command in the root of the repository:
+
+```sh
+pip3 install -r requirements.txt
+```
+
+If you run into conflicts with existing packages, a solution is to use a virtual Python environment. See the [Pelican installation page][3] for more details. These are quick commands, Linux flavor:
+
+```sh
+python3 -m venv env
+source env/bin/activate
+pip install -r requirements.txt
+```
+
+Once Pelican is installed you can convert your content into HTML via the pelican command (`content` is the default location to build from).
+
+```sh
+pelican
+```
+
+The above command will generate your site and save it in the `output/` folder using the solr theme and settings defined in `pelicanconf.py`
+
+You can also tell Pelican to watch for your modifications, instead of manually re-running it every time you want to see your changes. To enable this, run the pelican command with the `-r` or `--autoreload` option. On non-Windows environments, this option can also be combined with the `-l` or `--listen` option to simultaneously both auto-regenerate and serve the output through a builtin webserver on <http://localhost:8000>.
+
+```sh
+pelican --autoreload --listen
+```
+
+[1]: https://getpelican.com
+[2]: https://www.python.org/downloads/
+[3]: https://docs.getpelican.com/en/stable/install.html

pelicanconf.py

@@ -2,7 +2,7 @@
 # -*- coding: utf-8 -*- #
 from __future__ import unicode_literals
 from datetime import datetime, date
-#from checksumdir import dirhash
+from checksumdir import dirhash
 
 AUTHOR = 'Solr Developers'
 SITENAME = 'Apache Solr'
@@ -20,9 +20,7 @@
 
 # This string will be appended to all unversioned css and js resources to prevent caching surprises on edits.
 # The theme's htaccess file also sets a cache-control header with longer lifetime, if the v=XXXX query string is added.
-#STATIC_RESOURCE_SUFFIX = "?v=%s" % dirhash('themes/solr/static', 'sha1')[-8:]
-# SOLR-17339: Dynamic hash disabled until we can install custom modules like checksumdir. Bump this manually on changes
-STATIC_RESOURCE_SUFFIX = "?v=4dd59757"
+STATIC_RESOURCE_SUFFIX = "?v=%s" % dirhash('themes/solr/static', 'sha1')[-8:]
 
 PATH = 'content'
 
@@ -99,8 +97,7 @@
     'jinja2content',
     'regex_replace',
     'age_days_lt',
-# SOLR-17339: Vex disabled until https://github.com/apache/infrastructure-actions/pull/63 is merged
-#    'vex'
+    'vex'
 #    'md_inline_extension',
 ]
 

requirements.txt

@@ -1,12 +1,12 @@
 pelican~=4.9.1
 Markdown~=3.4
-#checksumdir~=1.2.0
+checksumdir~=1.2.0
 
 # Pelican plugins
 beautifulsoup4~=4.12.3
 # Dependencies of the vex plugin
-#jsonschema~=4.22.0
-#jsonref~=1.1.0
+jsonschema~=4.22.0
+jsonref~=1.1.0
 
 # Dev tools
 invoke~=2.2.0