Shiki is an excellent code highlighting library.
As we all know, code highlighting libraries contain numerous language parsers, which leads to slow loading and a waste of traffic due to loading unnecessary parsers.
In this section, we will implement on-demand loading of language parsers required by Shiki.
Dynamic Loading#
Before we start, we need to do some preparation work.
The definition of the Shiki component is similar to this.
```tsx import { bundledLanguages, getHighlighter } from 'shiki' import type { FC } from 'react'import { codeHighlighter } from './core'
const highlighter = await getHighlighter({
themes: [
import('shiki/themes/github-light.mjs'),
import('shiki/themes/github-dark.mjs'),
],
langs: Object.keys(bundledLanguages),
})
export const HighLighter: FC<{
lang: string
content: string
}> = async (props) => {
const { lang: language, content: value } = props
return (
<div
dangerouslySetInnerHTML={{
__html: codeHighlighter(highlighter, {
attrs: '',
code: value,
lang: language || '',
}),
}}
/>
)
}
</Tab>
<Tab label="page.tsx">
```tsx
const HighLighter = dynamic(() =>
import('../../components/Shiki/Shiki').then((mod) => mod.HighLighter),
)
export default () => {
return (
<HighLighter lang="javascript" content="console.log('Hello, World!')" />
)
}
In this way, the Shiki component is lazily loaded on the page, loading Shiki after the page is loaded. However, we reference all Shiki bundledLanguages, which results in loading the entire package even if we only use one language.
Although the page loading speed is slightly improved, the problem of wasted traffic is not solved.
On-Demand Loading of Language Parsers#
Shiki provides a method to dynamically load the required languages instead of loading all of them at once.
We can use getLoadedLanguages + getLoadedLanguages to encapsulate a component that implements this functionality.
Let's modify the above code:
'use client'
import { use } from 'react'
const codeHighlighterPromise = (async () => {
if (typeof window === 'undefined') return
const [{ getHighlighterCore }, getWasm, { codeHighlighter }] =
await Promise.all([
// There is a problem with Next.js 14.2.x, so we can only dynamically import here
import('shiki/core'),
import('shiki/wasm').then((m) => m.default),
import('./core'),
])
const core = await getHighlighterCore({
themes: [
import('shiki/themes/github-light.mjs'),
import('shiki/themes/github-dark.mjs'),
],
langs: [],// Do not load any languages here
loadWasm: getWasm,
})
return {
codeHighlighter: core,
fn: (o: { lang: string; attrs: string; code: string }) => {
return codeHighlighter(core, o)
},
}
})()
export const HighLighter: FC<{
lang: string
content: string
}> = (props) => {
const highlighter = use(codeHighlighterPromise) // Use `use` to wait for Promise resolved
const { lang: language, content: value } = props
use(
useMemo(async () => {
async function loadShikiLanguage(language: string, languageModule: any) {
const shiki = highlighter?.codeHighlighter
if (!shiki) return
if (!shiki.getLoadedLanguages().includes(language)) { // Check if it is loaded
await shiki.loadLanguage(await languageModule())
}
}
const { bundledLanguages } = await import('shiki/langs')
if (!language) return
const importFn = (bundledLanguages as any)[language]
if (!importFn) return
return loadShikiLanguage(language || '', importFn) // Dynamically load lang here
}, [highlighter?.codeHighlighter, language]),
)
const highlightedHtml = useMemo(() => {
return highlighter?.fn?.({
attrs: '',
code: value,
lang: language ? language.toLowerCase() : '',
})
}, [language, value, highlighter])
return (
<div
dangerouslySetInnerHTML={{
__html: highlightedHtml!,
}}
/>
)
}
In the above example, we use the use hook + Promise to dynamically load the required languages.
Finally, let's take a look at the number of requests.
As you can see, there are fewer requests now, only 17.
Some Final Thoughts#
In the Next.js App router, if we don't use Server Components to load Shiki, which means you have a need for dynamic rendering of code, you must use Client Components to encapsulate Shiki. However, the current Client Components in the App router do not support top-level await (and may never support it). In the example above, I use use to wait for the completion of the Shiki HighLighter, but what is the cost?
Shiki will never be rendered on the server side, which leads to Cumulative Layout Shift (CLS), which can be understood as page shaking. In the example above, we can only use the Shiki component in this way.
import { lazy, Suspense } from 'react'
const HighLighter = lazy(() => // Use lazy or dynamic, in Next.js 14.2.x, using dynamic will cause an error, but it is not affected in lower versions
import('../../components/Shiki/Shiki').then((mod) => ({
default: mod.HighLighter,
})),
)
export default () => {
return (
<Suspense fallback="Loading...">
<HighLighter lang="javascript" content="console.log('Hello, World!')" />
</Suspense>
)
}
The effect is that after refreshing, the Suspense fallback "Loading..." appears first, and then the rendering appears. This process is also completed in the browser.
To avoid CLS, the current solution is to make the fallback in Suspense occupy the space in advance. Although there will still be visual differences in the conversion of highlights, at least it can avoid the page shaking caused by CLS.
Note
The following solution is applicable when the component using your component can only be a Client Component. So in the example below, we assume that this Page cannot be a Server Component, because if the Page is a Server Component, this solution is meaningless. Because if it is a Server Component, we can render the code structure asynchronously rendered by Shiki in the Suspense fallback.
In fact, the actual business is much more complicated than this.
So when you need too many nested dynamic components, you should not use the Next.js App Router architecture, at least not now.
import { lazy, Suspense } from 'react'
const HighLighter = lazy(() =>
import('../../components/Shiki/Shiki').then((mod) => ({
default: mod.HighLighter,
})),
)
export default () => {
const code = `console.log('Hello, World!')`
return (
<Suspense
fallback={
<pre>
<code>{code}</code>
</pre>
}
>
<HighLighter lang="javascript" content={code} />
</Suspense>
)
}
Finally, thanks to antfu for providing the solution.
https://github.com/shikijs/shiki/issues/658
The above DEMO is simplified from the Shiro in the Shiki component.
Shiro is a simple but not simple personal website. It has even been featured in Ruan Lao Shi's newsletter. Why don't you come and try it out? 🥰
📜 A minimalist personal website embodying the purity of paper and freshness of snow.
This article is synchronized and updated to xLog by Mix Space
The original link is https://innei.in/posts/dev-story/shiki-dynamic-load-language