The search bar dropdown returns English search results directly rather than nearest language fallback
Closed, ResolvedPublicBUG REPORT
Actions

Assigned To

Authored By

	Hakimi97
	Mon, Jun 22, 9:27 AM

Project Tags

Referenced Files

	F91541876: image.png
	Thu, Jul 2, 6:25 AM

	F91541799: image.png
	Thu, Jul 2, 6:25 AM

	F91541657: image.png
	Thu, Jul 2, 6:25 AM

	F89938163: image.png
	Mon, Jun 22, 9:33 AM

	F89938095: image.png
	Mon, Jun 22, 9:33 AM

	F89934450: image.png
	Mon, Jun 22, 9:27 AM

	F89934062: image.png
	Mon, Jun 22, 9:27 AM

	F89933907: image.png
	Mon, Jun 22, 9:27 AM

Subscribers

Description

Steps to replicate the issue (include links if applicable):

Open https://www.wikidata.org/wiki/Wikidata:Main_Page , you need to log in since language selection is unavailable to the logged-out users
Click any of the language converter-supported language variant in the ULS (for example "中文（中国大陆）", "中文（简体）", "中文（臺灣）", "中文（繁體）") or language variant without language converter support in the ULS (for example "بهاس ملايو")
Type "Fish", or respective word (魚 for zh-Hant and zh-Hant-TW; 鱼 for zh-Hans and zh-Hans-CN; ايکن for ms-Arab)

What happens?: The dropdown option of the search bar all displayed English as the initial fallback, instead of following language fallback chain sequence as per description at https://www.mediawiki.org/wiki/Manual:Language

This situation does not happen when the languages selected were "中文" (zh) or "Bahasa Melayu" (ms) because both of them return the labels in respective language.

Suspected might be related to the format of language variant "langcode-scriptvariant(-geocode)", because different to the normal language code, the language variants have "-" in between, although this has not been verified.

What should have happened instead?:
The search bar content first should return respective language variant successfully, if failed then return the higher level of language fallback. For example: "zh" as the higher fallback default for zh-Hant, zh-Hans, zh-Hant-TW, zh-Hans-CN etc; and "ms" as the higher fallback default for ms-Arab. The English fallback "en" should act as the last resort for a language fallback.

Software version (on Special:Version page; skip for WMF-hosted wikis like Wikipedia):

Other information (browser name/version, screenshots, etc.):
This situation occurs regardless of switching parsers (Parsoid or Legacy Parser.php) or skins (Vector-2022 or Minerva)

Details

Related Changes in Gerrit:

	Subject	Author	Repo	Branch	Lines +/-
	Search: Use internal mw language code instead of bcp47	Jakob	mediawiki/extensions/Wikibase	master	+16 -2

Customize query in gerrit

Related Objects

Mentioned Here: T379135: [AUTO_LANGUAGE] does not use language fallbacks

Event Timeline

Hakimi97 created this task.Mon, Jun 22, 9:27 AM

Restricted Application added a project: Discovery-Search. · View Herald TranscriptMon, Jun 22, 9:27 AM

Restricted Application added a subscriber: • Aklapper. · View Herald Transcript

Hakimi97 updated the task description. (Show Details)Mon, Jun 22, 9:29 AM

Hakimi97 updated the task description. (Show Details)Mon, Jun 22, 9:33 AM

Hakimi97 updated the task description. (Show Details)

Hakimi97 updated the task description. (Show Details)Mon, Jun 22, 9:35 AM

Hakimi97 renamed this task from The search bar returns English search results directly rather than nearest language fallback in Wikidata to The search bar dropdown returns English search results directly rather than nearest language fallback.Mon, Jun 22, 9:38 AM

Hakimi97 added a subscriber: Taufik.

Hakimi97 updated the task description. (Show Details)Mon, Jun 22, 11:23 AM

Hakimi97 updated the task description. (Show Details)

Not sure if Content-Transform-Team should be tagged or not since this issue might somehow be related to Language Converter?

Or perhaps need to tag Language and Product Localization and I18n ?

First guess from @TJones: it might not work with language variants, e.g. Chinese works, but traditional Chinese doesn't. But we'll have a look.

pfischer set the point value for this task to 3.Mon, Jun 22, 3:25 PM

pfischer moved this task from Incoming to Ready for Dev on the Discovery-Search (2026.06.01 - 2026.07.03) board.

Since the search seems to work for "generic" or "default" versions of the language (e.g., 中文) but not the variants (e.g., 中文（中国大陆）), my hypothesis is that some relevant config is not being copied from the default language to the variant languages.

dcausse claimed this task.Thu, Jun 25, 4:00 PM

dcausse moved this task from Ready for Dev to In Progress on the Discovery-Search (2026.06.01 - 2026.07.03) board.

The problem seems to have appeared when enabling a new internal API (https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Wikibase/+/1276405). The cause seems related to some confusion with language codes set to bcp47 when some downstream search APIs expects an internal MW language code. For most language codes this is not an issue but for some there are ambiguities between the two, esp for zh-cn which is zh-Hans-CN when encoded as bcp47 but then cannot later be converted back to a mediawiki Language object (without calling LanguageCode::bcp47ToInternal).
I'm not familiar enough with the internal wikidata APIs to determine how to best fix this (cc @Jakob_WMDE).

dcausse removed dcausse as the assignee of this task.Fri, Jun 26, 9:22 AM

dcausse subscribed.

Hakimi97 added a project: Wikibase Action API (WPP).Fri, Jun 26, 10:49 AM

Perhaps this task is related to T379135

Hakimi97 added a project: Wikibase Reuse Team.Fri, Jun 26, 10:52 AM

@dcausse got it exactly right. We're mistakenly creating a bcp47 language code in SearchEntities.php where an internal MW language code is expected. The fix is as simple as replacing the current call to $this->getLanguageCode()->toBcp47Code() (which was wrong to begin with) with $this->getLanguage()->getCode(). I'll upload a patch for this with a regression test.