User:Novem Linguae/Essays/Docker tutorial for Windows (WSL)

Written from a Windows and VS Code perspective.

Some pessimistic advice

Expect to spend more time setting up your dev environment/toolchain than you do coding, until you've got it set up perfectly on all your computers, and you've mastered the ins and out of this work instruction. Can take months to become fluent. MediaWiki has a complicated toolchain.

Docker

Docker is a fancy XAMPP. It lets whatever codebase you're working on pick what OS, what version of PHP, what database, etc. to use instead of depending on whatever version of XAMPP you happened to install.

Why use Docker?

Here are some of the benefits:

• Solves "I dunno what your problem is, it works on my machine" issues
• You can configure your servers / cloud infrastructure to use the same Dockerfile as your devs, reducing issues

• Quickly spin up familiar infrastructure. You can just use your repo's Docker image instead of picking the right version of XAMPP, installing that, configuring XDebug, etc.
  • Did you know that Wikimedia production and continuous integration is on PHP 7.4 and Wikimedia devs are developing on PHP 8.1? Me neither, and it took me an hour of digging to figure this out. I could have just installed the MediaWiki Docker and had PHP 8.1 without thinking about it.
• Quickly spin up unfamiliar infrastructure. For example, if you're not familiar with python, you can run the Docker for a python repo and it'll install the correct version of python and pip (package manager) for you.
• Run legacy software. If you can get it working in Docker, it should work forever, even if it's old as dirt (e.g. running your old phpBB forum that uses PHP 5.6 and throws all sorts of warnings and errors on PHP 8)
• It's what's officially recommended in https://github.com/wikimedia/mediawiki/blob/master/DEVELOPERS.md, meaning it is probably the most common and best supported way to spin up mediawiki dev environments.
• Getting QUnit and Selenium tests to work without Docker has been a pain for me. In theory, following the official instructions at DEVELOPERS.md, which recommends installing Docker+Fresh, should get these to run.

How to set up Mediawiki Docker for Windows

• install Docker Desktop for Windows
• git clone "ssh://novemlinguae@gerrit.wikimedia.org:29418/mediawiki/core" - replace "novemlinguae" with your Gerrit username
• follow the official instructions at https://github.com/wikimedia/mediawiki/blob/master/DEVELOPERS.md
  • create .env file with default settings
  • docker compose up -d
  • docker compose exec mediawiki composer update
  • docker compose exec mediawiki /bin/bash /docker/install.sh - does initial configuration and database creation. assumes sqlite. if you already have a LocalSettings.php file and want to install mariadb, see below.
• VERY IMPORTANT FOR WINDOWS USERS: docker compose exec mediawiki chmod -R o+rwx cache/sqlite
• npm ci
• foreach (skin/extension):
  • git clone "ssh://novemlinguae@gerrit.wikimedia.org:29418/mediawiki/extensions/PageTriage" - replace "novemlinguae" with your Gerrit username, and replace "PageTriage" with the extension name
  • docker compose exec mediawiki composer update
  • npm ci
  • add wfLoadSkin( 'Vector' );, wfLoadExtension( 'PageTriage' ); , or similar to LocalSettings.php
• docker compose exec mediawiki php maintenance/run.php update - does database updates for skins and extensions

Step debugging

PHP step debugging: XDebug

Main articles: mw:MediaWiki-Docker/Configuration recipes/Xdebug and mw:MediaWiki-Docker/Configuration_recipes/Xdebug_config_for_VS_Code

Always run XDebug from the /mediawiki/ directory, not from an extension directory. According to the documentation, this is mandatory.

Add this to your .env file:

XDEBUG_CONFIG='mode=debug start_with_request=yes client_host=host.docker.internal client_port=9003 idekey=VSCODE' 
XDEBUG_MODE=debug

Replace your launch.json with this (may be optional, default config might work, maybe someday I'll test this):

{
	// Use IntelliSense to learn about possible attributes.
	// Hover to view descriptions of existing attributes.
	// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
	"version": "0.2.0",
	"configurations": [
		{
			"name": "Listen for XDebug",
			"type": "php",
			"request": "launch",
			"port": 9003,
			"pathMappings": {
			  "/var/www/html/w": "${workspaceFolder}"
			}
		},
		{
			"name": "Launch currently open script",
			"type": "php",
			"request": "launch",
			"program": "${file}",
			"cwd": "${fileDirname}",
			"port": 9003
		}
	]
}

JavaScript step debugging: Google Chrome devtools

• TODO: see if I can get this working in VS Code instead
• If you're having trouble setting a breakpoint (for example, the code you need is minified by ResourceLoader), add debugger; to your code.

VS Code

• If you're working on a MediaWiki extension or skin, open two windows: one for MediaWiki core, and one for the extension you're working on.
  • Run your step debugger in the MediaWiki core window (including setting breakpoints)
  • Do your coding work in the extension window. This will give you "search within repo", git, etc.
• Add this to your extension, in a file called .vscode/settings.json, so that MediaWiki core's libraries get imported and detected by PHP IntelliSense:

{
    "intelephense.environment.includePaths": [
        "../../"
    ]
}

Running tests

Add this to your .env file to get PHPUnit to stop outputting detailed debugging (recommended, else your unit test output is really noisy): PHPUNIT_LOGS=0

• how to run an extension's PHP unit tests
  • docker compose exec mediawiki composer phpunit:entrypoint - all
  • docker compose exec mediawiki composer phpunit:unit - tests in the /unit/ subfolder only
  • docker compose exec mediawiki composer phpunit:integration - tests in the /integration/ subfolder only
  • docker compose exec mediawiki composer phpunit:entrypoint --filter PageTriage - an extension's tests only
  • docker compose exec mediawiki composer phpunit:entrypoint extensions/PageTriage/tests/phpunit/ApiPageTriageActionTest.php - a specific test file only
• how to run an extension's Jest tests
• how to run an extension's QUnit tests
• how to run an extension's Selenium tests

SQL database

• how to install the database if you already have a LocalSettings.php file with correct database connection info, and a created database
  • harder than it should be. I've created a ticket. But in the meantime...
  • go into HeidiSQL, delete all the tables
  • rename your LocalSettings.php file to something else
  • re-run docker compose exec mediawiki php maintenance/run.php install, with all the correct CLI parameters
  • delete LocalSettings.php
  • rename your old LocalSettings.php back to LocalSettings.php
• how to update the database (installs SQL tables for extensions)
  • docker compose exec mediawiki php maintenance/run.php update
• how to drop all tables on a MariaDB
  • install HeidiSQL
  • drop the tables

SQLite or MariaDB?

• SQLite is the default. Pros and cons:
  • Pro - Keep your localhost database synced between computers, e.g. desktop and laptop, because the database is stored in the docker container in the /cache/ directory.
  • Pro - Easily clear the database by simply deleting the /cache/ directory.
  • Pro - Easy to set up a database viewer and editor, since you just need to point it to /cache/sqlite/my_wiki.sqlite
  • Con - Causes integration tests to fail for certain extensions such as PageTriage, likely due to atomicity issues.
  • Con - Different than Wikimedia production, which uses MariaDB
• MariaDB is an alternative. How to set it up:
  • mw:MediaWiki-Docker/Configuration recipes/Alternative databases#MariaDB (single database server)
    • follow this exactly. don't forget both the docker-compose.override.yml step and the maintenance/install.php step (copy pasting their custom string)
  • re-run docker compose exec mediawiki php maintenance/run.php update

Viewing and modifying the database: HeidiSQL

• to view/edit the SQL database, install HeidiSQL (download page)
• sqlite
  • point HeidiSQL at mediawiki/cache/sqlite
• mariadb
  • make sure your docker-compose/override.yml file has the following:

        ports:
          - 3306:3306

  • configure HeidiSQL with hostname = localhost, username = my_username, password = my_password, database = my_database

Miscellaneous

• File sizes
  • MediaWiki + skin + extension files is around 1.1 GB
  • Docker files are around ?? GB
• how to remote into Docker so that you don't have to add docker compose exec mediawiki to the start of every command, and so that you can cd around more easily
  • docker compose exec mediawiki bash
  • exit
• how to run an extension's maintenance script
  • docker compose exec mediawiki php extensions/PageTriage/maintenance/DeleteAfcStates.php
• restarts
  • any changes to the .env file require a restart of the Docker container: docker compose up -d

Troubleshooting

• Container mediawiki-mariadb-1: Error response from daemon: Ports are not available: exposing port TCP 0.0.0.0:3306 -> 0.0.0.0:0: listen tcp 0.0.0.0:3306: bind: Only one usage of each socket address (protocol/network address/port) is normally permitted.
  • Are you also running XAMPP? Close XAMPP, then go into Task Manager and terminate mysqld.exe.
• error during connect: This error may indicate that the docker daemon is not running.: Get "http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.24/containers/json?all=1&filters=%7B%22label%22%3A%7B%22com.docker.compose.project%3Dmediawiki%22%3Atrue%7D%7D&limit=0": open //./pipe/docker_engine: The system cannot find the file specified.
  • Start Docker Desktop, then try your CLI command again.

Windows Subsystem for Linux (WSL)

This will fix speed problems with Docker on Windows. Basically, you want to copy your mediawiki files into WSL's file system, then run Docker using the mediawiki files in WSL. The WSL file system is faster.

Install

• Install Docker
• Docker -> Settings -> General -> tick "Use the WSL 2 based engine"
• wsl --install -d ubuntu
  • When prompted, enter a username such as novemlinguae
  • When prompted, enter a password
  • When prompted, retype your password
• Docker -> Settings -> Resources -> WSL Integration -> tick "Ubuntu"

Move MediaWiki files from Windows file system to WSL file system

• If you don't git clone the files you need directly into WSL (see above), there's a couple different ways to put files in WSL
  • In Windows Explorer, copy and paste your mediawiki files into WSL by finding the shortcuts in the left menu, then clicking on Linux -> Ubuntu -> home -> novemlinguae, or
  • In Windows Explorer, copy and paste your mediawiki files into WSL from a regular folder into \\wsl.localhost\Ubuntu\home\novemlinguae, or
  • Type ubuntu to open a WSL terminal. Note that your pwd is /home/novemlinguae. Copy and paste your mediawiki files into WSL by typing cp -r /mnt/d/dropbox/code/mediawiki-docker/mediawiki ./mediawiki

Run Docker in a WSL terminal

• Open a PowerShell terminal (don't go into WSL/Ubuntu)
• cd \\wsl.localhost\Ubuntu\home\novemlinguae\mediawiki
• Use commands specified in #How to set up Mediawiki Docker for Windows and #Running tests, as needed

Load mediawiki files in WSL file system in VS Code

• Open VS Code
• Install the VS Code extension called "WSL"
• Go to File -> Open Folder... -> paste "\\wsl.localhost\Ubuntu\home\novemlinguae\mediawiki"
• A popup in the lower right will pop up suggesting you use VS Code running in WSL to open this. Click the link. It will install VS Code for WSL, then re-open VS Code for you.
• In the future, this will show up in File -> Open Recent, so you can quickly open it.

Using git in WSL

• todo: figure this out. "novemlinguae@gerrit.wikimedia.org: Permission denied (publickey)." Install git review and my SSH private key in WSL? Or figure out how to point git bash at WSL?