' + entry for entry in formatted])\n",
" display(HTML(f'
{formatted}
'))"
]
},
{
"cell_type": "markdown",
"id": "453c51fb-7e3c-47ed-8934-51b7627720cf",
"metadata": {},
"source": [
"# User Guide\n",
"\n",
"## Quick start\n",
"\n",
"You can install this extension from PyPI using:\n",
"\n",
"```bash\n",
"pip install -U jupyterlab-ui-profiler\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "2d371f3c-c257-486d-b7b3-3569474bbc85",
"metadata": {},
"source": [
"The ui-profiler can be started from Launcher's \"Other\" section:\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"id": "2c0efdc2-eeee-43c6-9f7a-4c3723b8c5be",
"metadata": {},
"source": [
"### Interface"
]
},
{
"cell_type": "markdown",
"id": "4c5e718a-f9b7-4310-9d38-350cae9cc9f8",
"metadata": {},
"source": [
"You can select multiple benchmarks (left) and multiple scenarios (right) at once. Configuration options for benchmarks and scenarios will show up when selected."
]
},
{
"cell_type": "markdown",
"id": "ce01e16a-f525-45cc-9500-0a93c8668ecf",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"id": "461a9943-8682-494b-a3bc-c679c0b2d11d",
"metadata": {},
"source": [
"After you selected the desired benchmarks and scenarios press ."
]
},
{
"cell_type": "markdown",
"id": "e0b3c8d2-04d0-47c1-88ad-96ca15a402fb",
"metadata": {},
"source": [
"For best results do not click, change focus or move your mouse after pressing \n",
" - most scenarios can be run when the browser window is blurred, but switching to CPU- or memory-heavy workload in another process may introduce additional variability\n",
" - most scenarios will **pause running** if you switch the tab (because `window.requestAnimationFrame` is used to synchronise execution with layout/paint events and it gets paused in background tabs)"
]
},
{
"cell_type": "markdown",
"id": "91cd38e2-f97f-45ce-83d0-ffeb86d1f2f9",
"metadata": {},
"source": [
"### Results\n",
"\n",
"The results will show in the area below the benchmark launcher:\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"id": "688f135a-3955-4182-9f59-0daff64b2015",
"metadata": {},
"source": [
"The summary of the benchmark run (left) and runtime environment (right) is shown at the top. You can hover over individual items to see more details (e.g. over \"Reference\" to see the list of times, over \"Browser\" to get detailed user agent information).\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"id": "944aee0f-4e5f-4b80-8d42-260beb90c746",
"metadata": {},
"source": [
"You can click on `> Options` to view the configuration with which the benchmark was run.\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"id": "4adac64d-8a38-43ed-8507-5148df93712b",
"metadata": {},
"source": [
"Tables with results can be sorted by clicking on the headers (click twice to reverse order), and columns can be resized by dragging edges of the headers."
]
},
{
"cell_type": "markdown",
"id": "b4ae19cb-3067-46ab-9c5a-0d5d92e5c9cf",
"metadata": {},
"source": [
"### Results history"
]
},
{
"cell_type": "markdown",
"id": "61f8e4f3-574b-4d4a-8209-831363ec8a26",
"metadata": {},
"source": [
"Results are stored in `.profile.json` files in `ui-profiler-results` directory (automatically created if needed) directly below the root directory of JupyterLab.\n",
"\n",
"If you want to remove a result from history you can remove the corresponding file directly; similarly if you want to preview a result generated on a different machine/instance you can copy it into the `ui-profiler-results` directory.\n",
"\n",
"The newest results are placed at the top of the list."
]
},
{
"cell_type": "markdown",
"id": "3a6048c9-fc68-4bc4-8556-952159cb8658",
"metadata": {},
"source": [
"## Measuring execution time"
]
},
{
"cell_type": "markdown",
"id": "895be424-c57f-47c0-a8f0-beee63c141ab",
"metadata": {},
"source": [
"> Note: The execution time is measured using `performance.now()` method which has reduced resolution in certain browsers/setups.\n",
">\n",
"> To enable high-precision timings in Firefox (79+) we send additional HTTP headers: `Cross-Origin-Opener-Policy: same-origin` and `Cross-Origin-Embedder-Policy: require-corp` which may interfere with user experience (for example restricting the download of images from third-party servers). If this is undesirable, disable the server extension with:\n",
"> ```\n",
"> jupyter server extension disable jupyterlab_ui_profiler\n",
"> ```"
]
},
{
"cell_type": "markdown",
"id": "ec07e4de-4401-4dd6-ae5a-47af8c018aa8",
"metadata": {},
"source": [
"### Available settings"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "70a4d8dc-7aac-417c-b0c1-e4f100a10210",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [],
"source": [
"show_settings('benchmark-execution')"
]
},
{
"cell_type": "markdown",
"id": "80aad3f9-6688-402d-bf7a-e7331ea2ff25",
"metadata": {},
"source": [
"### Interpretation"
]
},
{
"cell_type": "markdown",
"id": "0bb3207a-387a-4ca1-8aff-757709804be9",
"metadata": {},
"source": [
"In general performance timings are easily skewed towards higher values by external tasks such as garbage collection, activity of browser extensions or CPU utilisation by external processes. We recommend sorting by first quartile as default, with empirically second-best option being inter-quartile mean - but this varies by benchmark and scenario.\n",
"\n",
"For benchmarking of individual actions, box plot visualisation is provided. Hover over the circles, box lines, and whiskers to get details about the individual datapoints/statistics.\n",
"\n",
"\n",
"\n",
"The box represents 1st, 2nd (median) and 3rd quartile and whiskers correspond to minimum/maximum or 1.5 IQR away from the 1st/3rd quartile, whichever is smaller."
]
},
{
"cell_type": "markdown",
"id": "1dd0bfe5-06b9-49c3-983f-6eaca0b9e80b",
"metadata": {},
"source": [
"### Troubleshooting"
]
},
{
"cell_type": "markdown",
"id": "e0ab96ac-210d-4a7c-abb0-90329f49efb8",
"metadata": {},
"source": [
"If the timings in Firefox are of low resolution even though headers are sent properly, check your privacy settings, especially `privacy.resistFingerprinting`."
]
},
{
"cell_type": "markdown",
"id": "125f81d0-7776-471b-bea1-aed6f0ff274e",
"metadata": {},
"source": [
"## Benchmarking CSS styles"
]
},
{
"cell_type": "markdown",
"id": "7845834d-4189-416a-bfcf-5a41dce2e9d0",
"metadata": {},
"source": [
"Four benchmarks are provided for CSS styles:\n",
"- [Style Rules](style_rules): provides information on CSS rule performance by disabling rules one-by-one, but takes a long time to iterate over every single rule available\n",
"- [Style Sheets](style_sheets): disables entire CSS style sheets at a time; it is faster than disabling rules one-by-one, and more efficient when offending styles come from a single extension\n",
"- [Style Rule Groups](style_rule_groups): allows to triangulate the offending rules or group of rules by disabling them block-by-block and then calculating an average contribution of each rule based on the runtime of each block\n",
"- [Style Rule Usage](style_rule_usage): observes DOM modifications to calculate the frequency of usage for rules deemed relevant to the investigated scenario"
]
},
{
"cell_type": "markdown",
"id": "01fb02f9-84ae-4f04-bd43-f0da4b45ce53",
"metadata": {},
"source": [
"The results are presented in an interactive table which includes a set of columns shared between all CSS benchmarks (below) as well as benchmark-specific columns.\n",
"\n",
"The timings and statistics in table refer to execution time after disabling the rule/group of rules:\n",
"- `times`: measured execution times for each of the repeats\n",
"- `min`: fastest execution time\n",
"- `mean`: average execution time\n",
"- `IQM`: inter-quartile mean execution time\n",
"- `ΔIQM`: change in IQM\n",
"- `Q1`: first quartile of execution time\n",
"- `ΔQ1`: change in first quartile\n",
"\n",
"Other common columns are:\n",
"- `errors`: list of errors encountered during scenario execution (only present if any errors were encountered)\n",
"- `stylesheetIndex`: index of the style sheet among all stylesheets on the page\n",
"- `ruleIndex`: index of the rule within the source stylesheet\n",
"- `source`: the path to the source stylesheet if available. 📦 is an abbreviation for `node_modules` (locally developed/source extensions will not have a 📦 prefix)"
]
},
{
"cell_type": "markdown",
"id": "cce23034-0b20-45fa-87ef-3abe00f4cb56",
"metadata": {},
"source": [
"### Nomenclature"
]
},
{
"cell_type": "markdown",
"id": "bc5d01d8-c6d8-49ad-a870-f41025ee8336",
"metadata": {},
"source": [
"*Rules* are defined as groups of *properties*:"
]
},
{
"cell_type": "markdown",
"id": "70812f24-9a91-4d2f-98c8-78cf8985ac93",
"metadata": {},
"source": [
"```html\n",
"\n",
"\n",
"\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "885cb344-5eed-446d-90b0-a860dd4741b1",
"metadata": {},
"source": [
"### Limitations"
]
},
{
"cell_type": "markdown",
"id": "77a730e0-e4b8-472f-a8da-fa4a1a56b541",
"metadata": {},
"source": [
"Rules with pseudo-selectors and pseudo-classes in selector (e.g. `:hover`) cannot be invoked programatically, and therefore this ui-profiler cannot rule out performence bottlenecks resulting from such rules."
]
},
{
"cell_type": "markdown",
"id": "4906607f-7641-408c-8f0c-33c6671d4f6f",
"metadata": {},
"source": [
"(style_rules)=\n",
"### Style Rules"
]
},
{
"cell_type": "markdown",
"id": "eb736114-ad5c-455a-bffc-e07ed0d9db28",
"metadata": {},
"source": [
"#### Available settings"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fafb4b41-c397-43f6-9fae-f9b8cc3acac9",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [],
"source": [
"show_settings('benchmark-rule')"
]
},
{
"cell_type": "markdown",
"id": "30cba4a8-30c4-48c7-b355-b9bc88a53353",
"metadata": {},
"source": [
"#### Interpretation\n",
"\n",
"Negative Δ highlights rules which may be deteriorating performance.\n",
"\n",
"Due to multiple testing expect an enrichment of false positives which decreases as you increase the number of repeats. Empirically order of magnitude of 100 repeats is needed to minimise the false positives.\n",
"\n",
"Example result scenario for menu opening scenario with 10 repeats:\n",
"\n",
"\n",
"\n",
"This example demonstrates the issue of false positives well: only `.lm-MenuBar-itemMnemonic` and `.lm-Menu-itemShortcut` are relevant.\n",
"\n",
"Benchmark-specific columns:\n",
"- `bgMatches`: how many elements matched the rule at standby (as compared to during scenario execution); mostly useful to find too broad rules, or potentially unused rules with expensive selectors."
]
},
{
"cell_type": "markdown",
"id": "17c4bf40-129b-4847-b94d-3ccf1a17783c",
"metadata": {},
"source": [
"(style_sheets)=\n",
"### Style Sheets"
]
},
{
"cell_type": "markdown",
"id": "0ae1d50f-d568-4e79-95f4-b685747b1ed7",
"metadata": {},
"source": [
"#### Available settings"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "88cf1b92-da2e-48a8-b53a-4430faf74f8a",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [],
"source": [
"show_settings('benchmark-base')"
]
},
{
"cell_type": "markdown",
"id": "e5227b30-16cf-4a2e-82a5-677d07016bcd",
"metadata": {},
"source": [
"#### Interpretation"
]
},
{
"cell_type": "markdown",
"id": "d2f9f23c-d05f-4844-a966-dfcac5b49a55",
"metadata": {},
"source": [
"\n",
"\n",
"When a stylesheet is highlighted in this analysis, but no individual rule was highlighted in [Style Rules](style_rules) analysis, it may indicate a presence of interaction or synergy between rules."
]
},
{
"cell_type": "markdown",
"id": "f448ab8d-d94c-4038-b604-af89f09baa0b",
"metadata": {},
"source": [
"(style_rule_groups)=\n",
"### Style Rule Groups"
]
},
{
"cell_type": "markdown",
"id": "1dee117b-d11c-4a70-8c3d-29a20645468d",
"metadata": {},
"source": [
"#### Available settings"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7f693480-db6e-4b33-b230-b4bfb49aba2b",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [],
"source": [
"show_settings('benchmark-rule-group')"
]
},
{
"cell_type": "markdown",
"id": "067b02ee-f520-46f9-8556-8f200b79687a",
"metadata": {},
"source": [
"#### Interpretation"
]
},
{
"cell_type": "markdown",
"id": "72b46ca9-8591-49b8-b6ec-64bc1e4989e2",
"metadata": {},
"source": [
"Per-block results may highlight groups of styles degrading performance:"
]
},
{
"cell_type": "markdown",
"id": "7760ca2a-7acb-4ef0-aad1-19d0271f6ff4",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"id": "92328973-9e33-4cd3-95a2-02dde320e053",
"metadata": {},
"source": [
"Per-block results are easier to interpret with blocks of smaller size (higher granularity)."
]
},
{
"cell_type": "markdown",
"id": "fc843102-565f-4149-98f2-2cbddf544f93",
"metadata": {},
"source": [
"Per-rule averages may be useful to narrow down offending styles:"
]
},
{
"cell_type": "markdown",
"id": "1a34b762-7e73-46ab-b073-222dd7ba6c30",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"id": "07d7e1ea-124b-4238-80d0-5f6c5598b6f4",
"metadata": {},
"source": [
"In the above example exclusion of `.jp-JSONEditor*` or `.lm-Menu*` styles appears to improve performance. Given, the scenario (opening menu) the former is just coincidence (was selected in the same block more often) and latter indicates relevant styles."
]
},
{
"cell_type": "markdown",
"id": "5d6be35e-56fc-4cee-bae5-8e276d2fab03",
"metadata": {},
"source": [
"Benchmark-specific columns:\n",
"- `divisions`: into how many blocks was the list of styles divided in this run?\n",
"- `block`: which block of a particular run does this result refer to?\n",
"- `randomization`: which repeat with re-shuffled list of styles does this result refer to? 0th randomization is using the original order which often has similar styles next to each other (if original stylesheets are written in sensible order)"
]
},
{
"cell_type": "markdown",
"id": "0902551c-db3f-4b4a-8b6c-d85e895c1d20",
"metadata": {},
"source": [
"(style_rule_usage)=\n",
"### Style Rule Usage"
]
},
{
"cell_type": "markdown",
"id": "8c9cbbb4-fad8-43f1-a292-fad6d6fd9d76",
"metadata": {},
"source": [
"#### Available settings"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "465f9aed-589a-4887-b236-f0d58d2482c5",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [],
"source": [
"show_settings('benchmark-rule-usage')"
]
},
{
"cell_type": "markdown",
"id": "69fabadb-bd24-4075-aaf1-92cd9f6e3cc1",
"metadata": {},
"source": [
"#### Interpretation"
]
},
{
"cell_type": "markdown",
"id": "df2ab06e-474f-4f35-93ef-edbbc28e173b",
"metadata": {},
"source": [
"Benchmark-specific columns:\n",
"- `elementsSeen`: how many elements were seen on the entire page when executing the scenario.\n",
"- `elementsTouched`: how many elements were modified or in the subtree of a modified element when executing the scenario.\n",
"- `touchCount`: upper bound on how many times the rule matched an element (will be high for rules matching many elements, and for rules matching a single element that is repeatedly modified in the chosen scenario).\n",
"\n",
"Low number of `elementsSeen` suggest potentially unused rule. Negative Δ highlights rules which may be deteriorating performance."
]
},
{
"cell_type": "markdown",
"id": "b2649527-80e8-494a-a445-2850067d7bdd",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"id": "699efd02-fe94-4a63-a7c8-59d52fc8eb1c",
"metadata": {},
"source": [
"## Benchmarking JavaScript\n",
"\n",
"### At a glance\n",
"\n",
"- Benchmarking JavaScript code enables discovery of functions which\n",
" contribute to performance bottlenecks.\n",
"- This feature relies on [JS self-profiling API](https://github.com/WICG/js-self-profiling)\n",
" which as of 2022 is only available in Chromium-based browsers.\n",
" - The self-profiling API relies on sampling method to collect stack\n",
" (list of frames) at specified *sampling intervals*."
]
},
{
"cell_type": "markdown",
"id": "e3e47af7-08e5-48bb-91d4-d7a93c83f569",
"metadata": {},
"source": [
"### Available settings"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "088bd445-98cb-4b14-bedb-33b22d6dc45d",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [],
"source": [
"show_settings('benchmark-profile')"
]
},
{
"cell_type": "markdown",
"id": "bc2c20e2-4da1-4caa-b076-05b1a1d5d4d1",
"metadata": {},
"source": [
"Hints:\n",
"- for `micro` mode low number of repeats is often sufficient (e.g. 3), whereas `macro` mode is intended to be used with high number of repeats (e.g. 50).\n",
"- if `micro` mode does not catch the frames you want try increasing the number of repeats and scroll through the list of traces to find a repeat with satisfactory number of frames\n",
"- `macro` mode only averages the numbers in the table - the timeline is showing all repeats in order and this may be prohibitive to render if number of repeats is set to a very high number (e.g. 1000)"
]
},
{
"cell_type": "markdown",
"id": "29b1f0ba-66d0-4a7d-85f3-bef22eac4b36",
"metadata": {},
"source": [
"### Interpretation"
]
},
{
"cell_type": "markdown",
"id": "edbebc85-5025-430a-94b8-d23174df56f1",
"metadata": {},
"source": [
"> Note: when running on production version of JupyterLab names of functions and classes may be minimized. You can rebuild JupyterLab or use a development version to ease interpretation."
]
},
{
"cell_type": "markdown",
"id": "936f6e5f-e68b-48b7-9cc8-19fac991750c",
"metadata": {},
"source": [
"The micro mode provides the timeline for each individual run (repeat) of the scenario with stacks of frames corresponding to the execution stack:\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"id": "d54f9d00-6c39-47ff-b266-dff9870d0356",
"metadata": {},
"source": [
"In macro mode all runs are shown together chained in the timeline, but results table averages timings per function across repeats.\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"id": "7cda7f19-207c-4fbf-a7f8-6eae6d3b177e",
"metadata": {},
"source": [
"Legend:\n",
"- blue boxes denote native JavaScript code,\n",
"- green boxes indicate functions from JupyterLab, JupyterLab extensions or browser extensions,\n",
"- vertical dotted lines indicate boundaries for specific samples (local resolution of sampling)"
]
},
{
"cell_type": "markdown",
"id": "093ed1ed-c345-4be0-bfaf-c1cd409cad04",
"metadata": {},
"source": [
"Interactivity:\n",
"- hover over boxes to see execution time,\n",
"- you can zoom and pan to navigate larger traces,\n",
"- in the `micro` mode use the traces selector to choose the repeat which recorded all relevant frames (in `macro` mode there is only one trace),\n",
"- if the timeline fills the full height of the details section, you can resize it by grabbing bottom-right corner."
]
},
{
"cell_type": "markdown",
"id": "7f33895b-43e5-4b72-bd22-1b98750705d5",
"metadata": {},
"source": [
"Columns:\n",
"- `calls`: lower bound for the number of times given function was called (if the call was very fast and happened between samples it could not be recorded and hence would not be counted)\n",
"- `column`: location in the source file (resource): column\n",
"- `line`: location in the source file (resource): line"
]
},
{
"cell_type": "markdown",
"id": "a7318247-6097-4ed2-a732-8f3de446be1e",
"metadata": {},
"source": [
"### Troubleshooting"
]
},
{
"cell_type": "markdown",
"id": "decbd0e2-f821-44c6-af28-5d65ceb4a8a9",
"metadata": {},
"source": [
"To enable self-profiling we add `Document-Policy: js-profiling` header to the settings of jupyter-server (via `jupyterlab_ui_profiler` server extension).\n",
"\n",
"If the profiler is not available, ensure that the server extension is active using:\n",
"\n",
"\n",
"```\n",
"jupyter server extension list\n",
"```\n",
"\n",
"if it is active and you have just installed the extension, restart `jupyterlab` to ensure that it is loaded.\n",
"\n",
"If you use a replacement server (e.g. jupyverse) you will need to add the required header manually or switch to the default jupyter-server for the time of profiling."
]
},
{
"cell_type": "markdown",
"id": "73052b6b-24d6-4b91-812e-254a61f2ba90",
"metadata": {},
"source": [
"## Custom scenarios"
]
},
{
"cell_type": "markdown",
"id": "efb7e864-ddd7-45bd-a5a4-815c441bea77",
"metadata": {},
"source": [
"### Scenarios created via UI\n",
"\n",
"Starting with `v0.3` scenarios can be defined via UI as lists of Jupyter commands (use the \"Custom Scenario\" checkbox).\n",
"JupyterLab `4.5.0` or newer is required to render the forms for inputting arguments for built-in commands.\n",
"\n",
"`jupyterlab-ui-profiler` ships with a few commands making writing custom scenarios easier:\n",
"- `ui-profiler:wait-for-layout` - add this after an action that you want to benchmark to measure the actual time until the change is rendered on screen\n",
"- `ui-profiler:wait-for-selector` - use this to add wait conditions on the DOM state, that cannot be created by just awaiting for results of commands"
]
},
{
"cell_type": "markdown",
"id": "de361f64-5913-453f-8603-8624c92bf571",
"metadata": {},
"source": [
"### Programmatic scenarios\n",
"\n",
"You can define custom scenarios programmatically by creating a JupyterLab extension which consumes `IUIProfiler` token:\n",
"\n",
"```typescript\n",
"import { JupyterFrontEnd, JupyterFrontEndPlugin } from '@jupyterlab/application';\n",
"import { IScenario, IUIProfiler } from '@jupyterlab/ui-profiler';\n",
"\n",
"\n",
"class MyScenario implements IScenario {\n",
" id = 'myScenario';\n",
" name = 'My scenario';\n",
"\n",
" async run(): Promise {\n",
" console.log('Running!');\n",
" }\n",
"}\n",
"\n",
"export const plugin: JupyterFrontEndPlugin = {\n",
" id: '@my-organization/my-ui-profiler-extension:my-scenario',\n",
" autoStart: true,\n",
" requires: [IUIProfiler],\n",
" activate: (app: JupyterFrontEnd, profiler: IUIProfiler) => {\n",
" const myScenario = new MyScenario();\n",
" profiler.addScenario(myScenario);\n",
" }\n",
"}\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "f41ca671-fa3b-4aec-9985-7d2b1116d93c",
"metadata": {},
"source": [
"Please see [`tokens.ts` file](https://github.com/jupyterlab/ui-profiler/blob/main/src/tokens.ts) for the full documentation of `IScenario` interface."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.9"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
![\"UI](\"images/launcher.png\")
"
]
},
{
"cell_type": "markdown",
"id": "2c0efdc2-eeee-43c6-9f7a-4c3723b8c5be",
"metadata": {},
"source": [
"### Interface"
]
},
{
"cell_type": "markdown",
"id": "4c5e718a-f9b7-4310-9d38-350cae9cc9f8",
"metadata": {},
"source": [
"You can select multiple benchmarks (left) and multiple scenarios (right) at once. Configuration options for benchmarks and scenarios will show up when selected."
]
},
{
"cell_type": "markdown",
"id": "ce01e16a-f525-45cc-9500-0a93c8668ecf",
"metadata": {},
"source": [
"![\"UI](\"images/ui-profiler.png\")
"
]
},
{
"cell_type": "markdown",
"id": "461a9943-8682-494b-a3bc-c679c0b2d11d",
"metadata": {},
"source": [
"After you selected the desired benchmarks and scenarios press ."
]
},
{
"cell_type": "markdown",
"id": "e0b3c8d2-04d0-47c1-88ad-96ca15a402fb",
"metadata": {},
"source": [
"For best results do not click, change focus or move your mouse after pressing \n",
" - most scenarios can be run when the browser window is blurred, but switching to CPU- or memory-heavy workload in another process may introduce additional variability\n",
" - most scenarios will **pause running** if you switch the tab (because `window.requestAnimationFrame` is used to synchronise execution with layout/paint events and it gets paused in background tabs)"
]
},
{
"cell_type": "markdown",
"id": "91cd38e2-f97f-45ce-83d0-ffeb86d1f2f9",
"metadata": {},
"source": [
"### Results\n",
"\n",
"The results will show in the area below the benchmark launcher:\n",
"\n",
"![\"Result](\"images/ui-profiler-with-boxplot.png\")
"
]
},
{
"cell_type": "markdown",
"id": "688f135a-3955-4182-9f59-0daff64b2015",
"metadata": {},
"source": [
"The summary of the benchmark run (left) and runtime environment (right) is shown at the top. You can hover over individual items to see more details (e.g. over \"Reference\" to see the list of times, over \"Browser\" to get detailed user agent information).\n",
"\n",
"![\"Result](\"images/result-summary.png\")
"
]
},
{
"cell_type": "markdown",
"id": "944aee0f-4e5f-4b80-8d42-260beb90c746",
"metadata": {},
"source": [
"You can click on `> Options` to view the configuration with which the benchmark was run.\n",
"\n",
"![\"Result](\"images/result-options.png\")
"
]
},
{
"cell_type": "markdown",
"id": "4adac64d-8a38-43ed-8507-5148df93712b",
"metadata": {},
"source": [
"Tables with results can be sorted by clicking on the headers (click twice to reverse order), and columns can be resized by dragging edges of the headers."
]
},
{
"cell_type": "markdown",
"id": "b4ae19cb-3067-46ab-9c5a-0d5d92e5c9cf",
"metadata": {},
"source": [
"### Results history"
]
},
{
"cell_type": "markdown",
"id": "61f8e4f3-574b-4d4a-8209-831363ec8a26",
"metadata": {},
"source": [
"Results are stored in `.profile.json` files in `ui-profiler-results` directory (automatically created if needed) directly below the root directory of JupyterLab.\n",
"\n",
"If you want to remove a result from history you can remove the corresponding file directly; similarly if you want to preview a result generated on a different machine/instance you can copy it into the `ui-profiler-results` directory.\n",
"\n",
"The newest results are placed at the top of the list."
]
},
{
"cell_type": "markdown",
"id": "3a6048c9-fc68-4bc4-8556-952159cb8658",
"metadata": {},
"source": [
"## Measuring execution time"
]
},
{
"cell_type": "markdown",
"id": "895be424-c57f-47c0-a8f0-beee63c141ab",
"metadata": {},
"source": [
"> Note: The execution time is measured using `performance.now()` method which has reduced resolution in certain browsers/setups.\n",
">\n",
"> To enable high-precision timings in Firefox (79+) we send additional HTTP headers: `Cross-Origin-Opener-Policy: same-origin` and `Cross-Origin-Embedder-Policy: require-corp` which may interfere with user experience (for example restricting the download of images from third-party servers). If this is undesirable, disable the server extension with:\n",
"> ```\n",
"> jupyter server extension disable jupyterlab_ui_profiler\n",
"> ```"
]
},
{
"cell_type": "markdown",
"id": "ec07e4de-4401-4dd6-ae5a-47af8c018aa8",
"metadata": {},
"source": [
"### Available settings"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "70a4d8dc-7aac-417c-b0c1-e4f100a10210",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [],
"source": [
"show_settings('benchmark-execution')"
]
},
{
"cell_type": "markdown",
"id": "80aad3f9-6688-402d-bf7a-e7331ea2ff25",
"metadata": {},
"source": [
"### Interpretation"
]
},
{
"cell_type": "markdown",
"id": "0bb3207a-387a-4ca1-8aff-757709804be9",
"metadata": {},
"source": [
"In general performance timings are easily skewed towards higher values by external tasks such as garbage collection, activity of browser extensions or CPU utilisation by external processes. We recommend sorting by first quartile as default, with empirically second-best option being inter-quartile mean - but this varies by benchmark and scenario.\n",
"\n",
"For benchmarking of individual actions, box plot visualisation is provided. Hover over the circles, box lines, and whiskers to get details about the individual datapoints/statistics.\n",
"\n",
"![\"Boxplot\"](\"images/boxplot.png\")
\n",
"\n",
"The box represents 1st, 2nd (median) and 3rd quartile and whiskers correspond to minimum/maximum or 1.5 IQR away from the 1st/3rd quartile, whichever is smaller."
]
},
{
"cell_type": "markdown",
"id": "1dd0bfe5-06b9-49c3-983f-6eaca0b9e80b",
"metadata": {},
"source": [
"### Troubleshooting"
]
},
{
"cell_type": "markdown",
"id": "e0ab96ac-210d-4a7c-abb0-90329f49efb8",
"metadata": {},
"source": [
"If the timings in Firefox are of low resolution even though headers are sent properly, check your privacy settings, especially `privacy.resistFingerprinting`."
]
},
{
"cell_type": "markdown",
"id": "125f81d0-7776-471b-bea1-aed6f0ff274e",
"metadata": {},
"source": [
"## Benchmarking CSS styles"
]
},
{
"cell_type": "markdown",
"id": "7845834d-4189-416a-bfcf-5a41dce2e9d0",
"metadata": {},
"source": [
"Four benchmarks are provided for CSS styles:\n",
"- [Style Rules](style_rules): provides information on CSS rule performance by disabling rules one-by-one, but takes a long time to iterate over every single rule available\n",
"- [Style Sheets](style_sheets): disables entire CSS style sheets at a time; it is faster than disabling rules one-by-one, and more efficient when offending styles come from a single extension\n",
"- [Style Rule Groups](style_rule_groups): allows to triangulate the offending rules or group of rules by disabling them block-by-block and then calculating an average contribution of each rule based on the runtime of each block\n",
"- [Style Rule Usage](style_rule_usage): observes DOM modifications to calculate the frequency of usage for rules deemed relevant to the investigated scenario"
]
},
{
"cell_type": "markdown",
"id": "01fb02f9-84ae-4f04-bd43-f0da4b45ce53",
"metadata": {},
"source": [
"The results are presented in an interactive table which includes a set of columns shared between all CSS benchmarks (below) as well as benchmark-specific columns.\n",
"\n",
"The timings and statistics in table refer to execution time after disabling the rule/group of rules:\n",
"- `times`: measured execution times for each of the repeats\n",
"- `min`: fastest execution time\n",
"- `mean`: average execution time\n",
"- `IQM`: inter-quartile mean execution time\n",
"- `ΔIQM`: change in IQM\n",
"- `Q1`: first quartile of execution time\n",
"- `ΔQ1`: change in first quartile\n",
"\n",
"Other common columns are:\n",
"- `errors`: list of errors encountered during scenario execution (only present if any errors were encountered)\n",
"- `stylesheetIndex`: index of the style sheet among all stylesheets on the page\n",
"- `ruleIndex`: index of the rule within the source stylesheet\n",
"- `source`: the path to the source stylesheet if available. 📦 is an abbreviation for `node_modules` (locally developed/source extensions will not have a 📦 prefix)"
]
},
{
"cell_type": "markdown",
"id": "cce23034-0b20-45fa-87ef-3abe00f4cb56",
"metadata": {},
"source": [
"### Nomenclature"
]
},
{
"cell_type": "markdown",
"id": "bc5d01d8-c6d8-49ad-a870-f41025ee8336",
"metadata": {},
"source": [
"*Rules* are defined as groups of *properties*:"
]
},
{
"cell_type": "markdown",
"id": "70812f24-9a91-4d2f-98c8-78cf8985ac93",
"metadata": {},
"source": [
"```html\n",
"\n",
"\n",
"\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "885cb344-5eed-446d-90b0-a860dd4741b1",
"metadata": {},
"source": [
"### Limitations"
]
},
{
"cell_type": "markdown",
"id": "77a730e0-e4b8-472f-a8da-fa4a1a56b541",
"metadata": {},
"source": [
"Rules with pseudo-selectors and pseudo-classes in selector (e.g. `:hover`) cannot be invoked programatically, and therefore this ui-profiler cannot rule out performence bottlenecks resulting from such rules."
]
},
{
"cell_type": "markdown",
"id": "4906607f-7641-408c-8f0c-33c6671d4f6f",
"metadata": {},
"source": [
"(style_rules)=\n",
"### Style Rules"
]
},
{
"cell_type": "markdown",
"id": "eb736114-ad5c-455a-bffc-e07ed0d9db28",
"metadata": {},
"source": [
"#### Available settings"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fafb4b41-c397-43f6-9fae-f9b8cc3acac9",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [],
"source": [
"show_settings('benchmark-rule')"
]
},
{
"cell_type": "markdown",
"id": "30cba4a8-30c4-48c7-b355-b9bc88a53353",
"metadata": {},
"source": [
"#### Interpretation\n",
"\n",
"Negative Δ highlights rules which may be deteriorating performance.\n",
"\n",
"Due to multiple testing expect an enrichment of false positives which decreases as you increase the number of repeats. Empirically order of magnitude of 100 repeats is needed to minimise the false positives.\n",
"\n",
"Example result scenario for menu opening scenario with 10 repeats:\n",
"\n",
"\n",
"\n",
"This example demonstrates the issue of false positives well: only `.lm-MenuBar-itemMnemonic` and `.lm-Menu-itemShortcut` are relevant.\n",
"\n",
"Benchmark-specific columns:\n",
"- `bgMatches`: how many elements matched the rule at standby (as compared to during scenario execution); mostly useful to find too broad rules, or potentially unused rules with expensive selectors."
]
},
{
"cell_type": "markdown",
"id": "17c4bf40-129b-4847-b94d-3ccf1a17783c",
"metadata": {},
"source": [
"(style_sheets)=\n",
"### Style Sheets"
]
},
{
"cell_type": "markdown",
"id": "0ae1d50f-d568-4e79-95f4-b685747b1ed7",
"metadata": {},
"source": [
"#### Available settings"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "88cf1b92-da2e-48a8-b53a-4430faf74f8a",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [],
"source": [
"show_settings('benchmark-base')"
]
},
{
"cell_type": "markdown",
"id": "e5227b30-16cf-4a2e-82a5-677d07016bcd",
"metadata": {},
"source": [
"#### Interpretation"
]
},
{
"cell_type": "markdown",
"id": "d2f9f23c-d05f-4844-a966-dfcac5b49a55",
"metadata": {},
"source": [
"\n",
"\n",
"When a stylesheet is highlighted in this analysis, but no individual rule was highlighted in [Style Rules](style_rules) analysis, it may indicate a presence of interaction or synergy between rules."
]
},
{
"cell_type": "markdown",
"id": "f448ab8d-d94c-4038-b604-af89f09baa0b",
"metadata": {},
"source": [
"(style_rule_groups)=\n",
"### Style Rule Groups"
]
},
{
"cell_type": "markdown",
"id": "1dee117b-d11c-4a70-8c3d-29a20645468d",
"metadata": {},
"source": [
"#### Available settings"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7f693480-db6e-4b33-b230-b4bfb49aba2b",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [],
"source": [
"show_settings('benchmark-rule-group')"
]
},
{
"cell_type": "markdown",
"id": "067b02ee-f520-46f9-8556-8f200b79687a",
"metadata": {},
"source": [
"#### Interpretation"
]
},
{
"cell_type": "markdown",
"id": "72b46ca9-8591-49b8-b6ec-64bc1e4989e2",
"metadata": {},
"source": [
"Per-block results may highlight groups of styles degrading performance:"
]
},
{
"cell_type": "markdown",
"id": "7760ca2a-7acb-4ef0-aad1-19d0271f6ff4",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"id": "92328973-9e33-4cd3-95a2-02dde320e053",
"metadata": {},
"source": [
"Per-block results are easier to interpret with blocks of smaller size (higher granularity)."
]
},
{
"cell_type": "markdown",
"id": "fc843102-565f-4149-98f2-2cbddf544f93",
"metadata": {},
"source": [
"Per-rule averages may be useful to narrow down offending styles:"
]
},
{
"cell_type": "markdown",
"id": "1a34b762-7e73-46ab-b073-222dd7ba6c30",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"id": "07d7e1ea-124b-4238-80d0-5f6c5598b6f4",
"metadata": {},
"source": [
"In the above example exclusion of `.jp-JSONEditor*` or `.lm-Menu*` styles appears to improve performance. Given, the scenario (opening menu) the former is just coincidence (was selected in the same block more often) and latter indicates relevant styles."
]
},
{
"cell_type": "markdown",
"id": "5d6be35e-56fc-4cee-bae5-8e276d2fab03",
"metadata": {},
"source": [
"Benchmark-specific columns:\n",
"- `divisions`: into how many blocks was the list of styles divided in this run?\n",
"- `block`: which block of a particular run does this result refer to?\n",
"- `randomization`: which repeat with re-shuffled list of styles does this result refer to? 0th randomization is using the original order which often has similar styles next to each other (if original stylesheets are written in sensible order)"
]
},
{
"cell_type": "markdown",
"id": "0902551c-db3f-4b4a-8b6c-d85e895c1d20",
"metadata": {},
"source": [
"(style_rule_usage)=\n",
"### Style Rule Usage"
]
},
{
"cell_type": "markdown",
"id": "8c9cbbb4-fad8-43f1-a292-fad6d6fd9d76",
"metadata": {},
"source": [
"#### Available settings"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "465f9aed-589a-4887-b236-f0d58d2482c5",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [],
"source": [
"show_settings('benchmark-rule-usage')"
]
},
{
"cell_type": "markdown",
"id": "69fabadb-bd24-4075-aaf1-92cd9f6e3cc1",
"metadata": {},
"source": [
"#### Interpretation"
]
},
{
"cell_type": "markdown",
"id": "df2ab06e-474f-4f35-93ef-edbbc28e173b",
"metadata": {},
"source": [
"Benchmark-specific columns:\n",
"- `elementsSeen`: how many elements were seen on the entire page when executing the scenario.\n",
"- `elementsTouched`: how many elements were modified or in the subtree of a modified element when executing the scenario.\n",
"- `touchCount`: upper bound on how many times the rule matched an element (will be high for rules matching many elements, and for rules matching a single element that is repeatedly modified in the chosen scenario).\n",
"\n",
"Low number of `elementsSeen` suggest potentially unused rule. Negative Δ highlights rules which may be deteriorating performance."
]
},
{
"cell_type": "markdown",
"id": "b2649527-80e8-494a-a445-2850067d7bdd",
"metadata": {},
"source": [
""
]
},
{
"cell_type": "markdown",
"id": "699efd02-fe94-4a63-a7c8-59d52fc8eb1c",
"metadata": {},
"source": [
"## Benchmarking JavaScript\n",
"\n",
"### At a glance\n",
"\n",
"- Benchmarking JavaScript code enables discovery of functions which\n",
" contribute to performance bottlenecks.\n",
"- This feature relies on [JS self-profiling API](https://github.com/WICG/js-self-profiling)\n",
" which as of 2022 is only available in Chromium-based browsers.\n",
" - The self-profiling API relies on sampling method to collect stack\n",
" (list of frames) at specified *sampling intervals*."
]
},
{
"cell_type": "markdown",
"id": "e3e47af7-08e5-48bb-91d4-d7a93c83f569",
"metadata": {},
"source": [
"### Available settings"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "088bd445-98cb-4b14-bedb-33b22d6dc45d",
"metadata": {
"tags": [
"remove-input"
]
},
"outputs": [],
"source": [
"show_settings('benchmark-profile')"
]
},
{
"cell_type": "markdown",
"id": "bc2c20e2-4da1-4caa-b076-05b1a1d5d4d1",
"metadata": {},
"source": [
"Hints:\n",
"- for `micro` mode low number of repeats is often sufficient (e.g. 3), whereas `macro` mode is intended to be used with high number of repeats (e.g. 50).\n",
"- if `micro` mode does not catch the frames you want try increasing the number of repeats and scroll through the list of traces to find a repeat with satisfactory number of frames\n",
"- `macro` mode only averages the numbers in the table - the timeline is showing all repeats in order and this may be prohibitive to render if number of repeats is set to a very high number (e.g. 1000)"
]
},
{
"cell_type": "markdown",
"id": "29b1f0ba-66d0-4a7d-85f3-bef22eac4b36",
"metadata": {},
"source": [
"### Interpretation"
]
},
{
"cell_type": "markdown",
"id": "edbebc85-5025-430a-94b8-d23174df56f1",
"metadata": {},
"source": [
"> Note: when running on production version of JupyterLab names of functions and classes may be minimized. You can rebuild JupyterLab or use a development version to ease interpretation."
]
},
{
"cell_type": "markdown",
"id": "936f6e5f-e68b-48b7-9cc8-19fac991750c",
"metadata": {},
"source": [
"The micro mode provides the timeline for each individual run (repeat) of the scenario with stacks of frames corresponding to the execution stack:\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"id": "d54f9d00-6c39-47ff-b266-dff9870d0356",
"metadata": {},
"source": [
"In macro mode all runs are shown together chained in the timeline, but results table averages timings per function across repeats.\n",
"\n",
""
]
},
{
"cell_type": "markdown",
"id": "7cda7f19-207c-4fbf-a7f8-6eae6d3b177e",
"metadata": {},
"source": [
"Legend:\n",
"- blue boxes denote native JavaScript code,\n",
"- green boxes indicate functions from JupyterLab, JupyterLab extensions or browser extensions,\n",
"- vertical dotted lines indicate boundaries for specific samples (local resolution of sampling)"
]
},
{
"cell_type": "markdown",
"id": "093ed1ed-c345-4be0-bfaf-c1cd409cad04",
"metadata": {},
"source": [
"Interactivity:\n",
"- hover over boxes to see execution time,\n",
"- you can zoom and pan to navigate larger traces,\n",
"- in the `micro` mode use the traces selector to choose the repeat which recorded all relevant frames (in `macro` mode there is only one trace),\n",
"- if the timeline fills the full height of the details section, you can resize it by grabbing bottom-right corner."
]
},
{
"cell_type": "markdown",
"id": "7f33895b-43e5-4b72-bd22-1b98750705d5",
"metadata": {},
"source": [
"Columns:\n",
"- `calls`: lower bound for the number of times given function was called (if the call was very fast and happened between samples it could not be recorded and hence would not be counted)\n",
"- `column`: location in the source file (resource): column\n",
"- `line`: location in the source file (resource): line"
]
},
{
"cell_type": "markdown",
"id": "a7318247-6097-4ed2-a732-8f3de446be1e",
"metadata": {},
"source": [
"### Troubleshooting"
]
},
{
"cell_type": "markdown",
"id": "decbd0e2-f821-44c6-af28-5d65ceb4a8a9",
"metadata": {},
"source": [
"To enable self-profiling we add `Document-Policy: js-profiling` header to the settings of jupyter-server (via `jupyterlab_ui_profiler` server extension).\n",
"\n",
"If the profiler is not available, ensure that the server extension is active using:\n",
"\n",
"\n",
"```\n",
"jupyter server extension list\n",
"```\n",
"\n",
"if it is active and you have just installed the extension, restart `jupyterlab` to ensure that it is loaded.\n",
"\n",
"If you use a replacement server (e.g. jupyverse) you will need to add the required header manually or switch to the default jupyter-server for the time of profiling."
]
},
{
"cell_type": "markdown",
"id": "73052b6b-24d6-4b91-812e-254a61f2ba90",
"metadata": {},
"source": [
"## Custom scenarios"
]
},
{
"cell_type": "markdown",
"id": "efb7e864-ddd7-45bd-a5a4-815c441bea77",
"metadata": {},
"source": [
"### Scenarios created via UI\n",
"\n",
"Starting with `v0.3` scenarios can be defined via UI as lists of Jupyter commands (use the \"Custom Scenario\" checkbox).\n",
"JupyterLab `4.5.0` or newer is required to render the forms for inputting arguments for built-in commands.\n",
"\n",
"`jupyterlab-ui-profiler` ships with a few commands making writing custom scenarios easier:\n",
"- `ui-profiler:wait-for-layout` - add this after an action that you want to benchmark to measure the actual time until the change is rendered on screen\n",
"- `ui-profiler:wait-for-selector` - use this to add wait conditions on the DOM state, that cannot be created by just awaiting for results of commands"
]
},
{
"cell_type": "markdown",
"id": "de361f64-5913-453f-8603-8624c92bf571",
"metadata": {},
"source": [
"### Programmatic scenarios\n",
"\n",
"You can define custom scenarios programmatically by creating a JupyterLab extension which consumes `IUIProfiler` token:\n",
"\n",
"```typescript\n",
"import { JupyterFrontEnd, JupyterFrontEndPlugin } from '@jupyterlab/application';\n",
"import { IScenario, IUIProfiler } from '@jupyterlab/ui-profiler';\n",
"\n",
"\n",
"class MyScenario implements IScenario {\n",
" id = 'myScenario';\n",
" name = 'My scenario';\n",
"\n",
" async run(): Promise