^a^ainstitutetext: School of Physics, Nankai University, Tianjin 300071, China^b^binstitutetext: Deutsches Elektronen-Synchrotron DESY, Notkestr. 85, 22607 Hamburg, Germany

Searching for heavy neutral lepton and lepton number violation through VBS at high-energy muon colliders

Tong Li [email protected] a,b Chang-Yuan Yao [email protected] a Man Yuan [email protected]

Abstract

High-energy muon collider can play as an emitter of electroweak gauge bosons and thus leads to substantial vector boson scattering (VBS) processes. In this work, we investigate the production of heavy neutral lepton (HNL) $N$ and lepton number violation (LNV) signature through VBS at high-energy muon colliders. VBS induces LNV processes $W^{\pm}Z/\gamma\to\ell^{\pm}N\to\ell^{\pm}\ell^{\pm}W^{\mp}\to\ell^{\pm}\ell^{\pm}q\bar{q}^{\prime}$ with an on-shell HNL $N$ at $\mu^{+}\mu^{-}$ colliders. In analogy to neutrinoless double-beta decay with the HNL in t-channel, the LNV signature $W^{+}W^{+}\to\ell^{+}\ell^{+}$ can also happen via VBS at same-sign muon collider. They provide clean and robust LNV signatures to tell the nature of Majorana HNLs and thus have more advantageous benefits than direct $\mu\mu$ annihilation. We analyze the potential of searching for Majorana HNL and obtain the exclusion limits on mixing $V_{\ell N}$ . Based on this same-sign lepton signature, we also obtain the sensitivity of muon collider to the Weinberg operator.

^†^†preprint: DESY-23-092

1 Introduction

Neutrino oscillation experiments provide clear and compelling evidence that neutrinos have non-zero, but very small masses. The Standard Model (SM) includes only left-handed neutrino fields in the lepton doublets, and therefore predicts neutrino masses of exactly zero. Although one can certainly introduce right-handed (RH) neutrino field, the Yukawa coupling has to be tuned to a very small constant $y_{\nu}\lesssim 10^{-13}$ to accommodate the observed neutrino mass. A more economical way to generate neutrino mass in the SM content is through the so-called “Weinberg operator” Weinberg:1979sa

\displaystyle\ell_{L}\ell_{L}HH\;,

(1)

where $\ell_{L}$ and $H$ stand for the SM left-handed lepton doublet and the Higgs doublet, respectively. The price we pay here is to introduce a high-dimensional (dimension-5) operator and the violation of global lepton number symmetry. The minimal ultraviolet (UV) realization of this dimension-5 operator is the Type I Seesaw mechanism Minkowski:1977sc ; Yanagida:1979as ; GellMann:1980vs ; Glashow:1979nm ; Mohapatra:1979ia ; Shrock:1980ct ; Schechter:1980gr . In the minimal Type I Seesaw, the “sterile neutrinos” have the nature of Majorana fermions as they transform as singlet under the SM gauge group. The RH neutrinos possess a Majorana mass term $M_{R}$ and interact with SM leptons through a Yukawa coupling $y_{\nu}$ . The light neutrino masses are given by $m_{\nu}\sim y_{\nu}^{2}v^{2}/M_{R}$ with $v$ being the vacuum expectation value (vev) of SM Higgs. The heavy Majorana neutrinos can also arise in the Type III Seesaw mechanism with triplet fermionic multiplet Foot:1988aq . The eigenstates of RH neutrinos are usually referred as heavy neutral leptons (HNLs) which will be denoted as $N$ below. After mass mixing, one obtains an important mixing matrix $V_{\ell N}$ transiting heavy neutrinos to charged leptons in the mixed mass-flavor basis Atre:2009rg .

If $y_{\nu}$ is close to the electron Yukawa coupling $y_{e}\sim 10^{-6}$ in Type I Seesaw, the heavy Majorana neutrino receives a mass at TeV level and can be searched through lepton number violation (LNV) signature at high-energy colliders. This low-scale neutral lepton and the mixing matrix $V_{\ell N}$ have been searched at colliders. See Ref. Cai:2017mow and references therein for a review of LNV searches. The mass of HNL has been excluded up to about 1 TeV depending on the value of $|V_{\ell N}|^{2}\lesssim 0.1$ CMS:2018jxx ; CMS:2018iaf ; CMS:2022rqc ; ATLAS:2023tkz . The future FCC-hh can push the exclusion limit up to a few TeV Antusch:2016ejd ; Pascoli:2018heg ; Liu:2023gpt . One needs a machine with higher energy and integrated luminosity to further explore the measurement of $V_{\ell N}$ for the high mass region of HNL $N$ . High-energy muon colliders have recently received much attention in the community MICE:2019jkl ; Delahaye:2019omf ; Bartosik:2020xwr ; Han:2020uid ; Yin:2020afe ; AlAli:2021let ; Bose:2022obr ; Maltoni:2022bqs ; Narain:2022qud ; Li:2022kkc ; Li:2023ksw ; Chowdhury:2023imd . The high-energy options of muon collider can have multi-TeV center-of-mass (c.m.) energies and high integrated luminosity scaling with energy quadratically Delahaye:2019omf

\mathcal{L}=\left(\frac{\sqrt{s}}{10\ {\rm TeV}}\right)^{2}\times 10~{}\textrm{ab}^{-1}\;.

(2)

It thus provides an excellent opportunity to produce and discover HNLs in Type I and III Seesaw mechanisms.

Recently, there arose quite a few studies of searching for HNL at $\mu^{+}\mu^{-}$ colliders Mekala:2023diu ; Kwok:2023dck ; Li:2023tbx . They proposed that an HNL $N_{\ell}$ can be produced together with a light “active” neutrino $\nu_{\ell}$ from $\mu^{+}\mu^{-}$ collision with c.m. energy $\sqrt{s}=3$ or 10 TeV. The HNL then decays into a charged lepton $\ell^{\pm}$ and a $W$ gauge boson with hadronic decay. The production mode then becomes

\displaystyle\mu^{+}\mu^{-}\to N_{\ell}\bar{\nu}_{\ell}\to\ell^{\pm}\bar{\nu}_{\ell}q\bar{q}^{\prime}\;.

(3)

Despite of large production cross section of $\sigma(\mu^{+}\mu^{-}\to N_{\mu}\bar{\nu}_{\mu})\sim\mathcal{O}(10)$ pb through a $W$ exchange in t-channel Li:2023tbx , this signal channel cannot tell whether the HNL is Majorana fermion because of the missing neutrino in final states. One has to define a forward-backward symmetry in $N_{\ell}$ ’s decay pattern Li:2023tbx or other kinematic quantity Kwok:2023dck to distinguish Majorana and Dirac neutrinos. An alternative approach to search for Majorana neutrino is to consider the inverse $0\nu\beta\beta$ -like channel $\mu^{+}\mu^{+}\to W^{+}W^{+}$ Yang:2023ojm ; Jiang:2023mte which however relies on a same-sign muon collider.

In this work, we propose a clear way to search for heavy Majorana neutrino through LNV signature at $\mu^{+}\mu^{-}$ colliders. At high-energy muon colliders, the initial muon beams substantially emit electroweak (EW) gauge bosons under an approximately unbroken SM gauge symmetry. The gauge bosons are associated with very energetic muons or muon-neutrinos in the forward region with respect to the beam. The EW gauge bosons behave like initial state partons and lead to vector boson scattering (VBS) processes Costantini:2020stv ; AlAli:2021let ; Han:2021udl ; BuarqueFranzosi:2021wrv ; Ruiz:2021tdt . The VBS becomes an increasingly important mode as colliding energies go higher, due to the logarithmic enhancement from gauge boson radiation. Thus, a LNV signal can be produced through VBS

\displaystyle W^{\pm}Z/\gamma\to\ell^{\pm}N\to\ell^{\pm}\ell^{\pm}W^{\mp}\to\ell^{\pm}\ell^{\pm}q\bar{q}^{\prime}\;.

(4)

This signal is composed of same-sign charged leptons in final states and can thus clearly tell the Majorana nature of heavy neutrinos. It is exactly the smoking-gun signature searching for Majorana neutrino via Drell-Yan production $pp\to W^{\ast}\to\ell^{\pm}N$ Han:2006ip ; Atre:2009rg or $W\gamma$ VBS Dev:2013wba ; Alva:2014gxa at LHC. It can serve as a robust channel to identify LNV and Majorana neutrino suppose a discovery of HNL through the channel in Eq. (3). We will utilize the leading-order framework of electroweak parton distribution functions (EW PDFs) Han:2021udl ; Ruiz:2021tdt to calculate the HNL production through VBS. The detector simulation of LNV signal and SM backgrounds will be performed to analyze the search potentials on heavy Majorana neutrino and obtain the exclusion limit on mixing $V_{\ell N}$ .

In analogy to neutrinoless double-beta ( $0\nu\beta\beta$ ) decay, the VBS process can also induce LNV signature with the HNL in t-channel Dicus:1991fk ; Han:2006ip ; Atre:2009rg ; Fuks:2020att ; Fuks:2020zbm ; CMS:2022rqc ; Schubert:2022lcp . This VBS process happens at same-sign muon collider (e.g., $\mu^{+}\mu^{+}$ collider) and leads to the signal of same-sign charged leptons

\displaystyle W^{+}W^{+}\to\ell^{+}\ell^{+}\;.

(5)

This process provides a clean LNV signal compared with the inverse process $\mu^{+}\mu^{+}\to W^{+}W^{+}$ as there are no missing neutrinos and the suppression from two $W$ bosons’ decay branching fractions. We also analyze this LNV channel and explore the exclusion limit on mixing $V_{\ell N}$ . It can help to probe all charged lepton flavors and corresponding mixing elements rather than only $V_{\mu N}$ through $\mu^{+}\mu^{+}\to W^{+}W^{+}$ . In the absence of HNL, it was shown that this LNV process can also help to probe the Weinberg operator Fuks:2020zbm . We also evaluate the sensitivity of muon collider to the scale of Weinberg operator.

This paper is organized as follows. In Sec. 2, we first outline the HNLs in canonical Type I and Type III Seesaw models. Then we simulate the production of HNL and LNV signature via VBF processes at high-energy $\mu^{+}\mu^{-}$ colliders in Sec. 3. The results of projected sensitivity bounds for $V_{\ell N}$ are also given. In Sec. 4, we also analyze the LNV signature with the HNL in t-channel through VBS at same-sign muon collider. Finally, in Sec. 5 we summarize our conclusions.

2 Heavy neutral leptons in Seesaw mechanisms

The HNLs can be realized in canonical Type I and Type III Seesaw mechanisms. The neutrino Yukawa interactions in Type I Seesaw are

\displaystyle-\mathcal{L}_{Y}^{I}=Y^{D}_{\nu}\bar{\ell}_{L}\tilde{H}N_{R}+{\frac{1}{2}}\overline{(N^{c})_{L}}M_{R}N_{R}+\text{h.c.},

(6)

where $\tilde{H}=i\sigma_{2}H^{\ast}$ , $N_{R}$ denotes RH neutrino, and $M_{R}$ is the RH neutrino mass matrix. Once $H$ receives the SM vev $\langle H\rangle=v/\sqrt{2}$ , the Dirac mass term becomes $m_{D}=Y^{D}_{\nu}v/\sqrt{2}$ and the diagonalization of the neutrino mass matrix is

\displaystyle\mathbb{N}^{\dagger}\left(\begin{array}[]{cc}0&m_{D}\\ m_{D}^{T}&M_{R}\\ \end{array}\right)\mathbb{N}^{\ast}

\displaystyle=

\displaystyle\left(\begin{array}[]{cc}m_{\nu}^{\rm diag}&0\\ 0&M_{N}^{\rm diag}\\ \end{array}\right),

(11)

with the transformation of mass eigenstates as

\displaystyle\left(\begin{array}[]{c}\nu_{L}\\ (N^{c})_{L}\\ \end{array}\right)=\mathbb{N}\left(\begin{array}[]{c}\nu_{L}\\ (N^{c})_{L}\\ \end{array}\right)_{mass},\ \ \ \mathbb{N}=\left(\begin{array}[]{cc}U&V\\ X&Y\\ \end{array}\right)\;.

(18)

The mass eigensvalues of physical neutrinos are $m_{\nu}^{\rm diag}={\rm diag}(m_{1},m_{2},m_{3})$ and $M_{N}^{\rm diag}={\rm diag}(M_{1},\cdots)$ . With another matrix $E$ diagonalizing the charged lepton mass matrix, we have

\displaystyle E^{\dagger}U\equiv U_{\rm PMNS}\;,~{}~{}~{}E^{\dagger}V\equiv V_{\ell N}\;,

(19)

where $U_{\rm PMNS}$ is the approximate Pontecorvo-Maki-Nakagawa-Sakata (PMNS) neutrino mass mixing matrix. In particular, the matrix $V_{\ell N}$ transits heavy neutrinos to charged leptons Atre:2009rg . The active neutrino states are decomposed into a general number of massive eigenstates

\displaystyle\nu_{\ell}=\sum_{m=1}^{3}(U_{\rm PMNS})_{\ell m}\nu_{m}+\sum_{m^{\prime}=1}(V_{\ell N})_{\ell m^{\prime}}N_{m^{\prime}}^{c}\;,

(20)

where we assume that the HNL mass eigenstates are denoted as $N_{m^{\prime}}$ .

From Eq. (11), one can derive a relationship between diagonalized neutrino mass matrices and mixing matrices

\displaystyle U^{\ast}_{\rm PMNS}m_{\nu}^{\rm diag}U^{\dagger}_{\rm PMNS}+V^{\ast}_{\ell N}M_{N}^{\rm diag}V^{\dagger}_{\ell N}=0\ .

(21)

In the mixed mass-flavor basis, the matrix $V_{\ell N}$ determines the interactions between HNL and SM EW gauge bosons such as

	$\displaystyle\mathcal{L}_{\rm Type-I}$	$\displaystyle\supset$	$\displaystyle-{\frac{g}{\sqrt{2}}}W_{\mu}^{-}\sum_{\ell=e}^{\tau}\Big{(}\sum_{m=1}^{3}\bar{\ell}(U_{\rm PMNS})_{\ell m}\gamma^{\mu}P_{L}\nu_{m}+\sum_{m^{\prime}=1}\bar{\ell}(V_{\ell N})_{\ell m^{\prime}}\gamma^{\mu}P_{L}N^{c}_{m^{\prime}}\Big{)}+\text{h.c.}$
			$\displaystyle-\frac{g}{2\cos\theta_{W}}Z_{\mu}\sum_{\ell=e}^{\tau}\Big{(}\sum_{m=1}^{3}\bar{\nu}_{\ell}(U_{\rm PMNS})_{\ell m}\gamma^{\mu}P_{L}\nu_{m}+\sum_{m^{\prime}=1}\bar{\nu}_{\ell}(V_{\ell N})_{\ell m^{\prime}}\gamma^{\mu}P_{L}N^{c}_{m^{\prime}}\Big{)}+\text{h.c.}\;.$

The partial width of one HNL $N$ decay into charged lepton becomes

	$\displaystyle\Gamma(N\to\ell^{\pm}W^{\mp})$	$\displaystyle\equiv$	$\displaystyle\Gamma(N\to\ell^{+}W^{-})=\Gamma(N\to\ell^{-}W^{+})$		(23)
		$\displaystyle=$	$\displaystyle\frac{G_{F}}{8\sqrt{2}\pi}\|V_{\ell N}\|^{2}m_{N}(m_{N}^{2}+2m_{W}^{2})\left(1-\frac{m_{W}^{2}}{m_{N}^{2}}\right)^{2},$		(23)

and an asymptotic behavior holds when $m_{N}\gg m_{W},m_{Z},m_{h}$ .

\displaystyle\Gamma(N\to\sum_{\ell}\ell^{\pm}W^{\mp})\approx\Gamma(N\to\sum_{\nu}\nu Z+\bar{\nu}Z)\approx\Gamma(N\to\sum_{\nu}\nu h+\bar{\nu}h)\;,

(24)

where $\ell=e,\mu,\tau$ .

The Type III Seesaw introduces SU $(2)_{L}$ triplet leptons $\Sigma_{L}$ with zero hypercharge in addition to the SM fields Foot:1988aq

\displaystyle\Sigma_{L}=\left(\begin{array}[]{cc}\Sigma_{L}^{0}/\sqrt{2}&\Sigma^{+}_{L}\\ \Sigma_{L}^{-}&-\Sigma_{L}^{0}/\sqrt{2}\\ \end{array}\right)\ .

(27)

The Type III Seesaw Lagrangian is given by

\mathcal{L}_{\rm Type-III}={\rm Tr}\left[\overline{\Sigma_{L}}i\cancel{D}\Sigma_{L}\right]-\left(\frac{1}{2}{\rm Tr}\left[\overline{\Sigma_{L}^{c}}M_{\Sigma}\Sigma_{L}\right]+Y_{\Sigma}\overline{\ell_{L}}\tilde{H}\Sigma_{L}^{c}+\text{h.c.}\right)\;.

(28)

The Yukawa term in Eq. (28) induces both the neutrino mass mixing and a mass mixing between the charged SM leptons and the charged triplet leptons

\displaystyle\left(\begin{array}[]{cc}\overline{\nu_{L}^{c}}&\overline{\Sigma_{L}^{0c}}\\ \end{array}\right)\left(\begin{array}[]{cc}0&Y_{\Sigma}^{T}v_{0}/2\sqrt{2}\\ Y_{\Sigma}v_{0}/2\sqrt{2}&M_{\Sigma}/2\\ \end{array}\right)\left(\begin{array}[]{c}\nu_{L}\\ \Sigma_{L}^{0}\\ \end{array}\right)+\left(\begin{array}[]{cc}\overline{\ell_{R}}&\overline{\Sigma_{L}^{+c}}\\ \end{array}\right)\left(\begin{array}[]{cc}m_{\ell}&0\\ Y_{\Sigma}v_{0}&M_{\Sigma}\\ \end{array}\right)\left(\begin{array}[]{c}\ell_{L}\\ \Sigma_{L}^{-}\\ \end{array}\right)+\text{h.c.}\;.

(39)

After diagonalizing mass matrices by introducing unitary matrices, one obtains mass eigenvalues for neutrinos and charged leptons. The mixing between the SM charged leptons and triplet leptons follows the same relation in Eq. (21). The interactions between HNL and SM EW gauge bosons are also the same as those in Type I Seesaw.

In this section, we outline the theoretical motivations of the mixing matrix $V_{\ell N}$ between heavy neutrinos and charged leptons, i.e., Type I and Type III Seesaw models. In fact, the mixing parameter would scale like $V_{\ell N}^{2}\sim m_{\nu}/m_{N}$ and should be very small. We must state that we will assume the mixing angle $V_{\ell N}$ and the HNL mass $m_{N}$ as independent parameters in the following collider studies.

3 Heavy neutral lepton and lepton number violation through VBS at $\mu^{+}\mu^{-}$ collider

In this section, we investigate the search for Majorana HNL through VBS processes at future high-energy muon colliders. We simulate the LNV signatures from the decay of on-shell Majorana neutral lepton and obtain the sensitivity to the mixing parameters between HNL and SM charged leptons.

3.1 The VBS production of HNL and LNV signal

We propose to produce an HNL accompanied by a SM charged lepton $\ell_{1}^{\pm}N$ . Due to charge conservation, this combination can only be produced through VBS processes at muon colliders. After considering the decay of HNL $N$ , a LNV signature with $\Delta L=2$ can emerge

V_{i}~{}V_{j}\to\ell_{1}^{\pm}N\to\ell_{1}^{\pm}\ell_{2}^{\pm}q~{}\bar{q}^{\prime}\;,

(41)

where $\ell_{1},\ell_{2}=e,\mu,\tau$ and $V_{i},V_{j}$ denote SM EW gauge bosons. The decay channel of Majorana HNL is considered to be $N\to\ell_{2}^{\pm}W^{\mp}$ and the $W$ boson has subsequent decay into $q~{}\bar{q}^{\prime}$ . The Feynman diagrams of above LNV production are shown in Fig. 1. The panels (a) and (b) illustrate t-channel processes via exchanging a SM lepton and the panel (c) represents an s-channel process mediated by a virtual $W$ boson.

Refer to caption — Figure 1: The Feynman diagrams of processes $V_{i}~{}V_{j}\to\ell^{+}_{1}N\to\ell^{+}_{1}\ell^{+}_{2}q~{}\bar{q}^{\prime}$ at the muon collider.

For the initial gauge boson partons $V_{i}$ and $V_{j}$ , the VBS production cross section can be factorized as the product of the parton luminosity $d\mathcal{L}_{ij}/d\tau$ and the partonic cross section

	$\displaystyle\sigma(\ell^{+}\ell^{-}\to F+X)$	$\displaystyle=$	$\displaystyle\int^{1}_{\tau_{0}}d\tau\sum_{ij}\frac{d\mathcal{L}_{ij}}{d\tau}\sigma(V_{i}V_{j}\to F)\;,$
	$\displaystyle\frac{d\mathcal{L}_{ij}}{d\tau}$	$\displaystyle=$	$\displaystyle\frac{1}{1+\delta_{ij}}\int^{1}_{\tau}\frac{d\xi}{\xi}\Big{[}f_{i}(\xi,Q^{2})f_{j}(\frac{\tau}{\xi},Q^{2})+(i\leftrightarrow j)\Big{]}\;,$		(42)

where $F(X)$ denotes an exclusive final state (the underlying remnants), $f_{i(j)}(\xi,Q^{2})$ is the EW PDF for vector $V_{i(j)}$ with $Q=\sqrt{\hat{s}}/2$ being the factorization scale, $\tau_{0}=m_{F}^{2}/s$ and $\tau=\hat{s}/s$ . By summing over all gauge boson initial states, one can obtain the total cross section of the “inclusive” production processes. The scattering amplitude of the LNV process in Eq. (41) is proportional to $|V_{\ell_{1}N}V_{\ell_{2}N}|$ and the corresponding cross sections can be expressed as

	$\displaystyle\sigma~{}(V_{i}~{}V_{j}\to\ell_{1}^{\pm}N\to\ell_{1}^{\pm}\ell_{2}^{\pm}q\bar{q}^{\prime})$	$\displaystyle\approx\sigma~{}(V_{i}~{}V_{j}\to\ell_{1}^{\pm}N)\times{\rm BR}(N\to\ell_{2}^{\pm}q\bar{q}^{\prime})\times(2-\delta_{\ell_{1}\ell_{2}})$
		$\displaystyle\equiv\frac{\|V_{\ell_{1}N}V_{\ell_{2}N}\|^{2}}{\sum_{\ell=e,\mu,\tau}\|V_{\ell N}\|^{2}}\times\sigma_{0}\times(2-\delta_{\ell_{1}\ell_{2}})\;,$		(43)

where $\sigma_{0}$ represents the part of the cross section that is independent of the mixing parameters. We then define a new parameter which includes all the information of the mixing parameters Atre:2009rg

S_{\ell_{1}\ell_{2}}\equiv\frac{|V_{\ell_{1}N}V_{\ell_{2}N}|^{2}}{\sum_{\ell=e,\mu,\tau}|V_{\ell N}|^{2}}\;.

(44)

The parameter-independent cross section $\sigma_{0}=\sigma/S_{\ell_{1}\ell_{2}}$ for $\ell_{1}=\ell_{2}$ are shown in Fig. 2. The cross section of $\ell_{1}\neq\ell_{2}$ case is two times that shown here because of the exchange $\ell_{1}\leftrightarrow\ell_{2}$ in Eq. (43). If including the charge conjugation, another factor of two should be multiplied. For the following simulation of LNV signatures, we choose three cases of lepton flavor combinations in the final states with $\ell_{1},\ell_{2}=e,\mu$

S_{\mu\mu}=\frac{|V_{\mu N}|^{4}}{\sum_{\ell=e,\mu,\tau}|V_{\ell N}|^{2}},~{}S_{ee}=\frac{|V_{eN}|^{4}}{\sum_{\ell=e,\mu,\tau}|V_{\ell N}|^{2}},~{}S_{e\mu}=\frac{|V_{eN}V_{\mu N}|^{2}}{\sum_{\ell=e,\mu,\tau}|V_{\ell N}|^{2}}\;.

(45)

Next we consider the production of one single HNL denoted as $N$ .

3.2 The search for LNV signal at $\mu^{+}\mu^{-}$ collider

We employ the Universal FeynRules Output (UFO) Degrande:2011ua files of the model “HeavyN” Degrande:2016aje ; Pascoli:2018heg and pass the model files to MadGraph5_aMC@NLO Alwall:2014hca to generate the above LNV signal events at parton-level. The latest version of MadGraph5 has included the leading order framework of EW PDFs for the computation of the VBS processes at muon colliders Ruiz:2021tdt . We then use PYTHIA 8 Sjostrand:2014zea ; Bierlich:2022pfr for the parton shower of parton-level events. The detector card of the muon collider in Delphes 3 deFavereau:2013fsa is utilized to include the detector effects.

Our LNV signal is composed of two same-sign charged leptons plus jet(s), as shown in Fig. 1. We consider the following SM backgrounds

	$\displaystyle{\rm B_{1}}:~{}~{}V~{}V\to W^{\pm}W^{\pm}W^{\mp}\to\ell^{\pm}\ell^{\pm}\mathop{\nu_{\ell}}\limits^{(-)}\mathop{\nu_{\ell}}\limits^{(-)}q~{}\bar{q}^{\prime}\;,$
	$\displaystyle{\rm B_{2}}:~{}~{}V~{}V\to t~{}\overline{t}~{}W^{\pm}\to b~{}W^{+}\overline{b}~{}W^{-}W^{\pm}\to b~{}\bar{b}~{}\ell^{\pm}\ell^{\pm}\mathop{\nu_{\ell}}\limits^{(-)}\mathop{\nu_{\ell}}\limits^{(-)}+X\;,$		(46)

where $X$ denotes the decay products of the opposite-sign $W$ boson in ${\rm B_{2}}$ background, i.e., $X=\ell^{\mp}\mathop{\nu_{\ell}}\limits^{(-)}$ or $q\bar{q}^{\prime}$ . For ${\rm B_{1}}$ background, the intermediate states are three $W$ bosons which are produced through VBS process. The two same-sign $W$ bosons leptonically decay into a pair of same-sign leptons, and the other $W$ decays into dijet. Thus, the final states of ${\rm B_{1}}$ consist of two same-sign charged leptons, two jets and missing energy. In general, $b~{}(\bar{b})$ quark can be mistakenly identified as a light jet in the detector of the collider. Thus we also consider the ${\rm B_{2}}$ background in which a pair of $t~{}\bar{t}$ quarks associated with a $W$ boson are produced by VBS process. The $t~{}(\bar{t})$ quark decays into the $b~{}(\bar{b})$ quark and the $W^{\pm}$ boson. The same-sign $W$ bosons again decay into the lepton pairs and the other $W$ with opposite charge may decay into all possible final states. The events with $X=\ell^{\mp}\mathop{\nu_{\ell}}\limits^{(-)}$ can be efficiently reduced by imposing a soft $p_{T}$ cut on the opposite-sign lepton. Here we include them for a conservative background estimate.

We take the case of $\ell_{1}=\ell_{2}=\mu$ in Eq. (41) to illustrate the kinematic observables of our signal and backgrounds in Fig. 3 and Fig. 4. The benchmark masses of HNL are considered to be $m_{N}=200,~{}1000,~{}5000$ and $9000$ GeV for $\sqrt{s}=30$ TeV. From the decay of such heavy Majorana neutrino, the $W$ boson could be highly boosted and the produced dijet can be regarded as a single fat-jet $J$ Das:2017gke ; Bhardwaj:2018lma ; Das:2018usr in the detector of muon collider. The fat-jet is reconstructed via the “Valencia” algorithm with $R=1.2$ . We use $\mu_{1}$ to denote the muon produced in association with the HNL, and $\mu_{2}$ and $J$ (or $q\bar{q}^{\prime}$ ) are the decay products of HNL. We will explain how we distinguish them in our analysis below. The signal and backgrounds exhibit quite different kinematic distributions. We employ a series of selection cuts based on the distributions to suppress the SM backgrounds and enhance the significance.

•

We first select the generated events by requiring the number of same-sign muons and fat-jet, and identify the fat-jet as a $W$ boson

$\displaystyle N_{\mu}\geq 2,~{}N_{J}\geq 1\;;~{}~{}65~{}{\rm GeV}\leq m_{J}\leq 95~{}{\rm GeV}\;.$ (47)

If no fat-jet is found in the event, we also request two leading ordinary jets to satisfy the invariant mass window $65~{}{\rm GeV}\leq m_{j_{1}j_{2}}\leq 95~{}{\rm GeV}$ . But we find that taking into account these events would not change our results much. Thus, we only deal with the events with at least one fat-jet in the following analysis.

•

We employ some basic cuts for the muons in final states

\displaystyle p_{T}(\mu_{1})>50~{}{\rm GeV},~{}~{}p_{T}(\mu_{2})>m_{N}/4;~{}~{}|\eta(\mu_{1})|,|\eta(\mu_{2})|<2.5;~{}~{}\Delta R_{\mu_{1}\mu_{2}}>0.5\;.

(48)

The transverse momentum cut for $\mu_{1}$ is essential to avoid possible collinear divergence. As $\mu_{2}$ comes from the decay of $N$ , it could be very energetic depending on $m_{N}$ as shown in Fig. 3. We thus tighten up the kinematic cut for the transverse momentum of $\mu_{2}$ . It is worth mentioning that for the two muons in the final states, we separately combine them with $J$ . The muon that generates an invariant mass closer to $m_{N}$ is considered as $\mu_{2}$ .

•

For the final fat-jet $J$ coming from the decay of $N$ , its $p_{T}$ distribution is shown in the upper panels of Fig. 4. We employ the following kinematic cuts

$\displaystyle p_{T}(J)>m_{N}/4,~{}~{}|\eta(J)|<2.5\;.$ (49)
•

The presence of neutrinos in the final states of the backgrounds leads to missing energy, as shown in Eq. (46). For our signal without neutrinos, due to the effect of the energy resolution of detector, there still exists a distribution of small missing energy as shown in the middle panels of Fig. 4. As $m_{N}$ increases, this effect becomes progressively more significant. Thus, we only employ the missing energy cut for $m_{N}\leq 1000$ GeV

$\displaystyle\cancel{E}_{T}<30~{}{\rm GeV}~{}~{}~{}{\rm when}~{}~{}m_{N}\leq 1000~{}{\rm GeV}\;.$ (50)
•

We reconstruct Majorana HNL with $\mu_{2}$ and $J$ according to the invariant mass $m_{\mu_{2}J}$ shown in the lower panels of Fig. 4. Based on the signal distribution of the $m_{\mu_{2}J}$ , we employ an asymmetric mass window cut

$\displaystyle 0.8\times m_{N}<~{}m_{\mu_{2}J}~{}<1.1\times m_{N}\;.$ (51)

The background events can be efficiently suppressed after employing the above cuts. For illustration, we take $\sqrt{s}=30$ TeV for muon collider to present the signal and background cross sections after these cuts, as shown in Table 1.

sig. and bkgs	$\sigma/S_{\mu\mu}$ or	$N_{\mu}$ , $N_{J}$ , $m_{J}$	$\mu_{1},~{}\mu_{2}$	fat-jet $J$	$\cancel{E}_{T}$	$m_{\mu_{2}J}$
sig. and bkgs	$\sigma_{\rm B_{i}}$ [fb]	in Eq. (47)	in Eq. (48)	in Eq. (49)	in Eq. (50)	in Eq. (51)
$m_{N}=200$ GeV	7.67	1.17	0.90	0.89	0.80	0.80
$\rm B_{1}$	0.65	0.23	0.13	0.13	0.0081	8.6 $\times 10^{-4}$
$\rm B_{2}$	0.094	0.0054	0.0020	0.0020	2.0 $\times 10^{-4}$	4.0 $\times 10^{-5}$
$m_{N}=1000$ GeV	9.05	4.54	4.11	3.79	2.80	2.79
$\rm B_{1}$	0.65	0.23	0.048	0.036	0.0017	5.4 $\times 10^{-4}$
$\rm B_{2}$	0.094	0.0054	4.4 $\times 10^{-4}$	2.6 $\times 10^{-4}$	2.0 $\times 10^{-5}$	9.9 $\times 10^{-6}$
$m_{N}=5000$ GeV	4.56	2.70	2.41	2.19	–	2.17
$\rm B_{1}$	0.65	0.23	0.0066	0.0035	–	0.0012
$\rm B_{2}$	0.094	0.0054	1.2 $\times 10^{-5}$	0.0	–	0.0
$m_{N}=9000$ GeV	2.11	1.17	1.03	0.95	–	0.94
$\rm B_{1}$	0.65	0.23	0.0020	8.8 $\times 10^{-4}$	–	3.2 $\times 10^{-4}$
$\rm B_{2}$	0.094	0.0054	5.1 $\times 10^{-6}$	0.0	–	0.0

Table 1: The representative signal cross section

\sigma/S_{\mu\mu}

(in blue) and those for SM backgrounds (in black) at muon collider after selection cuts. For illustration, four benchmarks

m_{N}=200,~{}1000,~{}5000

and

9000

GeV are considered at muon collider with

\sqrt{s}=30

TeV.

After implementing the above analysis, we evaluate the sensitivity to the mixing constant $S_{\ell_{1}\ell_{2}}$ for Majorana HNL at muon colliders. For the significance, we use the following formula

\mathcal{S}=\frac{N_{\rm S}}{\sqrt{N_{\rm S}+N_{\rm B}}}\;,

(52)

where $N_{\rm S}$ and $N_{\rm B}=N_{\rm B_{1}}+N_{\rm B_{2}}$ are the event numbers of signal and backgrounds, respectively. They are given by

\displaystyle N_{\rm S}=\sigma_{0}~{}S_{\ell_{1}\ell_{2}}\times\epsilon_{\rm S}\times\mathcal{L}\;,~{}N_{\rm B_{1}}=\sigma_{{\rm B}_{1}}\times\epsilon_{{\rm B}_{1}}\times\mathcal{L}\;,~{}N_{\rm B_{2}}=\sigma_{{\rm B}_{2}}\times\epsilon_{{\rm B}_{2}}\times\mathcal{L}\;,

(53)

where $\epsilon_{\rm S,B_{1(2)}}$ represent the efficiencies of the above cuts and $\mathcal{L}$ denotes the integrated luminosity. From Eq. (2), we can get the corresponding integrated luminosity of the muon collider for different c.m. energies

	$\displaystyle\sqrt{s}$	$\displaystyle=~{}3,~{}10,~{}~{}30~{}\left[{\rm TeV}\right]\;,$
	$\displaystyle\mathcal{L}$	$\displaystyle=~{}1,~{}10,~{}~{}90~{}\left[{\rm ab}^{-1}\right]\;.$		(54)

We obtain the 2 $\sigma$ exclusion limits to $S_{\ell_{1},\ell_{2}}$ with $\ell_{1},\ell_{2}=e,\mu$ versus $m_{N}$ at muon colliders with $\sqrt{s}=3$ (red), 10 (green) and 30 (blue) TeV, as shown in solid lines in Fig. 5.

The quantity $S_{\ell\ell}$ is equal to the commonly used $|V_{\ell N}|^{2}$ under the assumption of only one non-vanishing flavor element for the matrix $V_{\ell N}$ . Under this assumption, we use $|V_{\mu N}|^{2}$ or $|V_{eN}|^{2}$ as the frame labels of the y-axis in the upper and lower-left panels of Fig. 5. One can see that, for $m_{N}$ well below $\sqrt{s}$ , the value of $|V_{\mu N}|^{2}$ can be probed as low as $4\times 10^{-3}$ ( $3\times 10^{-4}$ ) [ $5\times 10^{-5}$ ] for $\sqrt{s}=3~{}(10)~{}[30]$ TeV. This potential is worse than that through $\mu^{+}\mu^{-}\to N_{\mu}\bar{\nu}_{\mu}$ channel Kwok:2023dck ; Li:2023tbx , because the latter happens via a t-channel exchange of a $W$ boson with huge production cross section. The projected sensitivity bound of $|V_{eN}|^{2}$ is similar to that of $|V_{\mu N}|^{2}$ . The difference comes from the efficiency of identifying $e$ or $\mu$ from the collider simulation. However, this $|V_{eN}|^{2}$ exclusion limit is stronger than that through $\mu^{+}\mu^{-}\to N_{e}\bar{\nu}_{e}$ channel for $\sqrt{s}=10$ TeV. This is because the electron HNL can only be produced from $\mu^{+}\mu^{-}$ annihilation via s-channel and its production cross section is exceeded by VBS production at high energies. Moreover, this LNV process through VBS can provide a clean signal of $e^{\pm}\mu^{\pm}$ and an exclusion limit on quantity $S_{e\mu}$ as shown in the lower-right panel of Fig. 5.

4 Lepton number violation through VBS at $\mu^{+}\mu^{+}$ collider

4.1 The probe of heavy neutral lepton

In this section we consider the LNV signature through VBS at same-sign muon collider

\displaystyle W^{+}W^{+}\to\ell^{+}\ell^{+}\;.

(55)

This VBS process is mediated by either light neutrino mass eigenstates $\nu_{m}$ or the HNL $N_{m^{\prime}}$ in t-channel and is analogous to neutrinoless double- $\beta$ decay. Their amplitudes are proportional to $m_{\nu_{m}}$ and $m_{N_{m^{\prime}}}$ , respectively. We thus only consider the process mediated by one HNL $N$ in t-channel as shown by the Feynman diagram in Fig. 6.

The cross section of this VBS process can be expressed as

\sigma~{}(W^{+}W^{+}\to\ell_{1}^{+}\ell_{2}^{+})\equiv|V_{\ell_{1}N}V_{\ell_{2}N}|^{2}\times\sigma_{0}\;,

(56)

where $\sigma_{0}$ is the “bare” cross section independent of the mixing parameters. The values of cross section $\sigma_{0}$ for $\ell_{1}=\ell_{2}$ are shown in Fig. 7 which are one-half of those for $\ell_{1}\neq\ell_{2}$ because of the average of identical final states. This LNV signal is purely composed of two same-sign charged leptons. We consider three primary SM backgrounds

	$\displaystyle{\rm B_{1}}:~{}~{}\mu^{+}\mu^{+}\to\mu^{+}\mu^{+}\;,$
	$\displaystyle{\rm B_{2}}:~{}~{}\mu^{+}\mu^{+}\to\mu^{+}\mu^{+}\nu~{}\bar{\nu}\;,$
	$\displaystyle{\rm B_{3}}:~{}~{}\mu^{+}\mu^{+}\to W^{+}W^{+}\bar{\nu}_{\mu}~{}\bar{\nu}_{\mu}\to\ell^{+}\ell^{+}\nu_{\ell}~{}\nu_{\ell}~{}\bar{\nu}_{\mu}~{}\bar{\nu}_{\mu}\;,$

where $\nu~{}(\bar{\nu})$ donates the summation of neutrinos (anti-neutrinos) with three flavors. The backgrounds ${\rm B_{1}}$ and ${\rm B_{2}}$ only appear for the situation with $\ell_{1}=\ell_{2}=\mu$ , and we also take this case as an example to illustrate the distributions of signal and backgrounds in Fig. 8. Corresponding to other flavor combinations of charged leptons in our signal, only background ${\rm B_{3}}$ applies. According to the different kinematic distributions of signal and backgrounds, we adopt a series of selection cuts to suppress the backgrounds and enhance the significance.

•

Firstly, we require the number of same-sign muons in the generated events, and employ some basic cuts for the muons in final states

N_{\mu}\geq 2,~{}~{}p_{T}(\mu_{1},\mu_{2})>50~{}{\rm GeV},~{}~{}|\eta(\mu_{1})|,|\eta(\mu_{2})|<2.5,~{}~{}\Delta R_{\mu_{1}\mu_{2}}>0.5\;.

(57)

•

The leading transverse momentum $p_{T}^{\rm max}(\mu)$ of the final muons is shown in the first row of Fig. 8. We employ some relatively conservative kinematic cuts

$p_{T}^{\rm max}(\mu)~{}>~{}100~{}(200)~{}[500]~{}{\rm GeV}\;,$ (58)

for $\sqrt{s}=3~{}(10)~{}[30]$ TeV. They help to reduce the background ${\rm B_{3}}$ in which the charged leptons are from two $W$ bosons’ leptonic decay.
•

The cosine of the angle between two muons in final states $\cos\theta_{\ell\ell}$ exhibits very different behaviors between signal and SM backgrounds, as shown in the second row of Fig. 8. Thus, we employ the following cut

$\cos\theta_{\ell\ell}<0.2\;.$ (59)
•

For backgrounds ${\rm B_{2}}$ and ${\rm B_{3}}$ , the presence of neutrinos leads to missing energy, as shown in the third row of Fig. 8. We can employ the cut for the missing energy to reduce the backgrounds ¹¹1The cut on the $p_{T}$ of the sub-leading charged lepton could also help to reduce the backgrounds at the LHC ATLAS:2023tkz . We find that it is worse than the missing energy cut here.

$\cancel{E}_{T}<20~{}(50)~{}[150]~{}{\rm GeV}\;,$ (60)

for $\sqrt{s}=3~{}(10)~{}[30]$ TeV.
•

The final muons in background ${\rm B_{1}}$ carry all the collision energy. Its invariant mass leads to a significantly different distribution from the signal and other backgrounds, as shown in the last row of Fig. 8. According to this feature, we can employ the following invariant mass cut to extremely reduce the background ${\rm B_{1}}$

$\displaystyle m_{\ell\ell}~{}$ $\displaystyle<~{}2000~{}(5000)~{}[15000]~{}{\rm GeV}\;,$ (61)

for $\sqrt{s}=3~{}(10)~{}[30]$ TeV.

We take the case of $\sqrt{s}=30$ TeV for muon collider to illustrate the cross sections of the signal and backgrounds after the above series of cuts, as shown in Table 2.

sig. and bkgs	$\sigma/\|V_{\mu N}\|^{4}$ or	$N_{\mu}$ and basic cuts	$p_{T}^{\rm max}(\mu)$	${\rm cos}~{}\theta_{\ell\ell}$	$\cancel{E}_{T}$	$m_{\ell\ell}$
sig. and bkgs	$\sigma_{\rm B_{i}}$ [pb]	in Eq. (57)	in Eq. (58)	in Eq. (59)	in Eq. (60)	in Eq. (61)
$m_{N}=200$ GeV	0.072	0.070	0.041	0.030	0.027	0.026
$m_{N}=1000$ GeV	0.64	0.62	0.57	0.44	0.36	0.36
$m_{N}=5000$ GeV	1.99	1.95	1.94	1.77	1.01	0.97
$m_{N}=9000$ GeV	1.84	1.84	1.83	1.75	0.83	0.76
$\rm B_{1}$	0.055	0.051	0.051	0.051	0.011	0.0
$\rm B_{2}$	0.0046	0.0042	0.0042	0.0024	4.5 $\times 10^{-5}$	1.3 $\times 10^{-5}$
$\rm B_{3}$	0.011	0.0018	4.1 $\times 10^{-4}$	2.8 $\times 10^{-4}$	9.9 $\times 10^{-6}$	9.9 $\times 10^{-6}$

Table 2: The representative signal cross section

\sigma/|V_{\mu N}|^{4}

(in blue) and those for SM backgrounds (in black) at muon collider after selection cuts. For illustration, four benchmarks

m_{N}=200,~{}1000,~{}5000

and

9000

GeV are considered at muon collider with

\sqrt{s}=30

TeV.

We adopt the same significance formula in Eq. (52) here with $N_{\rm B}=N_{\rm B_{1}}+N_{\rm B_{2}}+N_{\rm B_{3}}$ . The $N_{\rm S}$ and $N_{{\rm B}_{i}}$ ( $i=1,2,3$ ) are given by

\displaystyle N_{\rm S}=\sigma_{0}~{}|V_{\ell_{1}N}V_{\ell_{2}N}|^{2}\times\epsilon_{\rm S}\times\mathcal{L}\;,~{}N_{\rm B_{i}}=\sigma_{{\rm B}_{i}}\times\epsilon_{{\rm B}_{i}}\times\mathcal{L}\;.

(62)

The corresponding integrated luminosity of the muon collider for different c.m. energies are shown in Eq. (54). We obtain the sensitivity of muon colliders to the mixing parameter $|V_{\ell_{1}N}V_{\ell_{2}N}|$ for Majorana HNL. The $2\sigma$ exclusion limits to $|V_{\mu N}|^{2}$ , $|V_{eN}|^{2}$ and $|V_{eN}V_{\mu N}|$ versus the mass of Majorana HNL are shown in dash-dotted lines in Fig. 5. In contrast to the $\ell_{1}^{\pm}\ell_{2}^{\pm}+{\rm jet(s)}$ signature losing sensitivity near the energy threshold, this t-channel LNV signal can probe much heavier HNL.

4.2 The probe of Weinberg operator

Finally, we comment on the probe of dimension-5 Weinberg operator in the absence of HNL. The above signature of same-sign charged leptons can also be used to probe the Weinberg operator Fuks:2020zbm ; ATLAS:2023tkz

\displaystyle{C_{5}^{\ell\ell^{\prime}}\over\Lambda}H\cdot\bar{\ell}_{L}^{c}\ell^{\prime}_{L}\cdot H\;,

(63)

where $\ell_{L}^{T}=(\nu_{\ell},\ell)$ . After EW symmetry breaking and the SM Higgs gains its vev $v$ , this Weinberg operator generates a Majorana neutrino mass in the flavor basis

\displaystyle m_{\ell\ell^{\prime}}={C_{5}^{\ell\ell^{\prime}}v^{2}\over\Lambda}\,,

(64)

which appears as the mass of the intermediate fermion in the same-sign $W$ scattering. It was shown in Ref. Fuks:2020zbm that the parton-level cross section of this same-sign $W$ scattering process in this model is

\displaystyle\sigma(W^{+}W^{+}\to\ell\ell^{\prime})\approx{2-\delta_{\ell\ell^{\prime}}\over 18\pi}\Big{|}{C_{5}^{\ell\ell^{\prime}}\over\Lambda}\Big{|}^{2}\;.

(65)

The cross section is proportional to an overall scaling factor of $|C_{5}^{\ell\ell^{\prime}}/\Lambda|^{2}$ .

Next we follow the approach in Ref. Fuks:2020zbm to simulate the probe of Weinberg operator for $\ell=\ell^{\prime}=\mu$ at high-energy muon colliders. It was verified that a reasonable simulation using the UFO for Weinberg operator (called “SMWeinberg”) has to be performed with $\Lambda/|C_{5}^{\ell\ell^{\prime}}|\geq 200$ TeV Fuks:2020zbm . We adopt the same selection strategies mentioned in last subsection and obtain the number of signal events denoted by $N_{\rm S}^{200}$ for a benchmark with $\Lambda=200$ TeV and $C_{5}^{\mu\mu}=1$ . According to Eq. (65), the number of signal events is given by

\displaystyle N_{\rm S}=N_{\rm S}^{200}|C_{5}^{\mu\mu}|^{2}\Big{(}{200~{}{\rm TeV}\over\Lambda}\Big{)}^{2}\;.

(66)

By requiring $2\sigma$ significance, we can obtain the sensitivity to the scale $\Lambda/|C_{5}^{\mu\mu}|$ . We find that the sensitivity bound of muon collider to the scale is

\displaystyle\Lambda/|C_{5}^{\mu\mu}|\lesssim 9.3~{}(23.0)~{}[47.7]~{}{\rm TeV}\;,

(67)

for $\sqrt{s}=3~{}(10)~{}[30]$ TeV. This is equivalent to the limit of effective $\mu\mu$ Majorana mass as

\displaystyle m_{\mu\mu}\gtrsim 6.5~{}(2.6)~{}[1.3]~{}{\rm GeV}\;.

(68)

5 Conclusion

In this work, we propose the search for Majorana HNLs and LNV signatures through VBS processes at high-energy muon colliders. At high energies, the muon beams radiate substantial EW gauge bosons and can play as “gauge boson colliders”. The VBS processes lead to clear LNV signatures so as to tell the nature of Majorana HNLs and thus provide more advantageous benefits than direct $\mu\mu$ annihilation. We use the method of EW PDFs to calculate the VBS production of HNL at both $\mu^{+}\mu^{-}$ and $\mu^{+}\mu^{+}$ colliders, and analyze the produced LNV signatures. At $\mu^{+}\mu^{-}$ collider, the scattering of gauge bosons $W^{\pm}Z/\gamma$ induces the associated production of HNL $N$ and charged lepton $\ell^{\pm}$ . At same-sign muon collider, the $W^{+}W^{+}$ scattering can result in a $0\nu\beta\beta$ -like LNV process $W^{+}W^{+}\to\ell^{+}\ell^{+}$ with the HNL in t-channel.

We perform the detector simulation of LNV signal and SM backgrounds. The search potentials on heavy Majorana neutrino are analyzed and the exclusion limits on the mixing of HNL $N$ and charged lepton $V_{\ell N}$ are obtained. We find the following conclusions:

•

The VBS processes can provide smoking-gun LNV signatures $\ell_{1}^{\pm}\ell_{2}^{\pm}+{\rm jet(s)}$ and $\ell_{1}^{+}\ell_{2}^{+}$ at $\mu^{+}\mu^{-}$ collider and same-sign muon collider, respectively.
•

The probing potential of $|V_{\mu N}|^{2}$ through $W^{\pm}Z/\gamma\to\mu^{\pm}\mu^{\pm}+{\rm jet(s)}$ signature is worse than that from annihilation channel $\mu^{+}\mu^{-}\to N_{\mu}\bar{\nu}_{\mu}$ . However, the exclusion limit on $|V_{eN}|^{2}$ from VBS channel is stronger than that through the $\mu^{+}\mu^{-}$ annihilation for $\sqrt{s}=10$ TeV or above.
•

At same-sign muon collider, compared with the $\ell^{\pm}\ell^{\pm}+{\rm jet(s)}$ signature, the $0\nu\beta\beta$ -like LNV signal can help to probe much heavier HNL. The mixings other than $|V_{\mu N}|$ can be probed through VBS as well, which is contrary to the process $\mu^{+}\mu^{+}\to W^{+}W^{+}$ . Based on this signature, we also estimate the sensitivity of muon collider to the scale of Weinberg operator.
•

The above LNV signatures can both be employed to look for the combinations of different charged lepton flavors and probe the mixing $|V_{\ell_{1}N}V_{\ell_{2}N}|$ with $\ell_{1}\neq\ell_{2}$ .

Acknowledgements.

We would like to thank Richard Ruiz for useful discussion. T.L. is supported by the National Natural Science Foundation of China (Grants No. 11975129, 12035008) and “the Fundamental Research Funds for the Central Universities”, Nankai University (Grant No. 63196013). C.Y.Y. is supported in part by the Grants No. NSFC-11975130, No. NSFC-12035008, No. NSFC-12047533, the Helmholtz-OCPC International Postdoctoral Exchange Fellowship Program, the National Key Research and Development Program of China under Grant No. 2017YFA0402200, the China Postdoctoral Science Foundation under Grant No. 2018M641621, and the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy — EXC 2121 “Quantum Universe” — 390833306.

References

(1) S. Weinberg, Baryon and Lepton Nonconserving Processes, Phys. Rev. Lett. 43 (1979) 1566–1570.
(2) P. Minkowski, $\mu\to e\gamma$ at a Rate of One Out of $10^{9}$ Muon Decays?, Phys. Lett. B 67 (1977) 421–428.
(3) T. Yanagida, Horizontal gauge symmetry and masses of neutrinos, Conf. Proc. C 7902131 (1979) 95–99.
(4) M. Gell-Mann, P. Ramond and R. Slansky, Complex Spinors and Unified Theories, Conf. Proc. C 790927 (1979) 315–321, [1306.4669].
(5) S. L. Glashow, The Future of Elementary Particle Physics, NATO Sci. Ser. B 61 (1980) 687.
(6) R. N. Mohapatra and G. Senjanovic, Neutrino Mass and Spontaneous Parity Nonconservation, Phys. Rev. Lett. 44 (1980) 912.
(7) R. E. Shrock, General Theory of Weak Leptonic and Semileptonic Decays. 1. Leptonic Pseudoscalar Meson Decays, with Associated Tests For, and Bounds on, Neutrino Masses and Lepton Mixing, Phys. Rev. D 24 (1981) 1232.
(8) J. Schechter and J. W. F. Valle, Neutrino Masses in SU(2) x U(1) Theories, Phys. Rev. D 22 (1980) 2227.
(9) R. Foot, H. Lew, X. G. He and G. C. Joshi, Seesaw Neutrino Masses Induced by a Triplet of Leptons, Z. Phys. C 44 (1989) 441.
(10) A. Atre, T. Han, S. Pascoli and B. Zhang, The Search for Heavy Majorana Neutrinos, JHEP 05 (2009) 030, [0901.3589].
(11) Y. Cai, T. Han, T. Li and R. Ruiz, Lepton Number Violation: Seesaw Models and Their Collider Tests, Front. in Phys. 6 (2018) 40, [1711.02180].
(12) CMS collaboration, A. M. Sirunyan et al., Search for heavy Majorana neutrinos in same-sign dilepton channels in proton-proton collisions at $\sqrt{s}=13$ TeV, JHEP 01 (2019) 122, [1806.10905].
(13) CMS collaboration, A. M. Sirunyan et al., Search for heavy neutral leptons in events with three charged leptons in proton-proton collisions at $\sqrt{s}=$ 13 TeV, Phys. Rev. Lett. 120 (2018) 221801, [1802.02965].
(14) CMS collaboration, Probing heavy Majorana neutrinos and the Weinberg operator through vector boson fusion processes in proton-proton collisions at $\sqrt{s}$ = 13 TeV, 2206.08956.
(15) ATLAS collaboration, G. Aad et al., Search for Majorana neutrinos in same-sign $WW$ scattering events from $pp$ collisions at $\sqrt{s}=13$ TeV, 2305.14931.
(16) S. Antusch, E. Cazzato and O. Fischer, Sterile neutrino searches at future $e^{-}e^{+}$ , $pp$ , and $e^{-}p$ colliders, Int. J. Mod. Phys. A 32 (2017) 1750078, [1612.02728].
(17) S. Pascoli, R. Ruiz and C. Weiland, Heavy neutrinos with dynamic jet vetoes: multilepton searches at $\sqrt{s}=14$ , 27, and 100 TeV, JHEP 06 (2019) 049, [1812.08750].
(18) W. Liu, J. Li, Z. Chen and H. Sun, Probing Heavy Neutrinos at the LHC from Fat-jet using Machine Learning, 2303.15920.
(19) MICE collaboration, M. Bogomilov et al., Demonstration of cooling by the Muon Ionization Cooling Experiment, Nature 578 (2020) 53–59, [1907.08562].
(20) J. P. Delahaye, M. Diemoz, K. Long, B. Mansoulié, N. Pastrone, L. Rivkin et al., Muon Colliders, 1901.06150.
(21) N. Bartosik et al., Detector and Physics Performance at a Muon Collider, JINST 15 (2020) P05001, [2001.04431].
(22) T. Han, Y. Ma and K. Xie, High energy leptonic collisions and electroweak parton distribution functions, Phys. Rev. D 103 (2021) L031301, [2007.14300].
(23) W. Yin and M. Yamaguchi, Muon g-2 at a multi-TeV muon collider, Phys. Rev. D 106 (2022) 033007, [2012.03928].
(24) H. Al Ali et al., The muon Smasher’s guide, Rept. Prog. Phys. 85 (2022) 084201, [2103.14043].
(25) T. Bose et al., Report of the Topical Group on Physics Beyond the Standard Model at Energy Frontier for Snowmass 2021, 2209.13128.
(26) F. Maltoni et al., TF07 Snowmass Report: Theory of Collider Phenomena, 2210.02591.
(27) M. Narain et al., The Future of US Particle Physics - The Snowmass 2021 Energy Frontier Report, 2211.11084.
(28) T. Li, H. Qin, C.-Y. Yao and M. Yuan, Probing heavy triplet leptons of the type-III seesaw mechanism at future muon colliders, Phys. Rev. D 106 (2022) 035021, [2205.04214].
(29) T. Li, C.-Y. Yao and M. Yuan, Revealing the origin of neutrino masses through the Type II Seesaw mechanism at high-energy muon colliders, JHEP 03 (2023) 137, [2301.07274].
(30) T. A. Chowdhury, A. Jueid, S. Nasri and S. Saad, Probing Zee-Babu states at Muon Colliders, 2306.01255.
(31) K. Mękała, J. Reuter and A. F. Żarnecki, Optimal search reach for heavy neutral leptons at a muon collider, Phys. Lett. B 841 (2023) 137945, [2301.02602].
(32) T. H. Kwok, L. Li, T. Liu and A. Rock, Searching for Heavy Neutral Leptons at A Future Muon Collider, 2301.05177.
(33) P. Li, Z. Liu and K.-F. Lyu, Heavy neutral leptons at muon colliders, JHEP 03 (2023) 231, [2301.07117].
(34) J.-L. Yang, C.-H. Chang and T.-F. Feng, The leptonic di-flavor and di-number violation processes at high energy $\mu^{\pm}\mu^{\pm}$ colliders, 2302.13247.
(35) R. Jiang, T. Yang, S. Qian, Y. Ban, J. Li, Z. You et al., Searching for Majorana Neutrinos at a Same-Sign Muon Collider, 2304.04483.
(36) A. Costantini, F. De Lillo, F. Maltoni, L. Mantani, O. Mattelaer, R. Ruiz et al., Vector boson fusion at multi-TeV muon colliders, JHEP 09 (2020) 080, [2005.10289].
(37) T. Han, S. Li, S. Su, W. Su and Y. Wu, Heavy Higgs bosons in 2HDM at a muon collider, Phys. Rev. D 104 (2021) 055029, [2102.08386].
(38) D. Buarque Franzosi et al., Vector boson scattering processes: Status and prospects, Rev. Phys. 8 (2022) 100071, [2106.01393].
(39) R. Ruiz, A. Costantini, F. Maltoni and O. Mattelaer, The Effective Vector Boson Approximation in high-energy muon collisions, JHEP 06 (2022) 114, [2111.02442].
(40) T. Han and B. Zhang, Signatures for Majorana neutrinos at hadron colliders, Phys. Rev. Lett. 97 (2006) 171804, [hep-ph/0604064].
(41) P. S. B. Dev, A. Pilaftsis and U.-k. Yang, New Production Mechanism for Heavy Neutrinos at the LHC, Phys. Rev. Lett. 112 (2014) 081801, [1308.2209].
(42) D. Alva, T. Han and R. Ruiz, Heavy Majorana neutrinos from $W\gamma$ fusion at hadron colliders, JHEP 02 (2015) 072, [1411.7305].
(43) D. A. Dicus, D. D. Karatas and P. Roy, Lepton nonconservation at supercollider energies, Phys. Rev. D 44 (1991) 2033–2037.
(44) B. Fuks, J. Neundorf, K. Peters, R. Ruiz and M. Saimpert, Majorana neutrinos in same-sign $W^{\pm}W^{\pm}$ scattering at the LHC: Breaking the TeV barrier, Phys. Rev. D 103 (2021) 055005, [2011.02547].
(45) B. Fuks, J. Neundorf, K. Peters, R. Ruiz and M. Saimpert, Probing the Weinberg operator at colliders, Phys. Rev. D 103 (2021) 115014, [2012.09882].
(46) J. L. Schubert and O. Ruchayskiy, Neutrinoless double-beta decay at colliders: interference between Majorana states, 2210.11294.
(47) C. Degrande, C. Duhr, B. Fuks, D. Grellscheid, O. Mattelaer and T. Reiter, UFO - The Universal FeynRules Output, Comput. Phys. Commun. 183 (2012) 1201–1214, [1108.2040].
(48) C. Degrande, O. Mattelaer, R. Ruiz and J. Turner, Fully-Automated Precision Predictions for Heavy Neutrino Production Mechanisms at Hadron Colliders, Phys. Rev. D 94 (2016) 053002, [1602.06957].
(49) J. Alwall, R. Frederix, S. Frixione, V. Hirschi, F. Maltoni, O. Mattelaer et al., The automated computation of tree-level and next-to-leading order differential cross sections, and their matching to parton shower simulations, JHEP 07 (2014) 079, [1405.0301].
(50) T. Sjöstrand, S. Ask, J. R. Christiansen, R. Corke, N. Desai, P. Ilten et al., An introduction to PYTHIA 8.2, Comput. Phys. Commun. 191 (2015) 159–177, [1410.3012].
(51) C. Bierlich et al., A comprehensive guide to the physics and usage of PYTHIA 8.3, 2203.11601.
(52) DELPHES 3 collaboration, J. de Favereau, C. Delaere, P. Demin, A. Giammanco, V. Lemaître, A. Mertens et al., DELPHES 3, A modular framework for fast simulation of a generic collider experiment, JHEP 02 (2014) 057, [1307.6346].
(53) A. Das, P. Konar and A. Thalapillil, Jet substructure shedding light on heavy Majorana neutrinos at the LHC, JHEP 02 (2018) 083, [1709.09712].
(54) A. Bhardwaj, A. Das, P. Konar and A. Thalapillil, Looking for Minimal Inverse Seesaw scenarios at the LHC with Jet Substructure Techniques, J. Phys. G 47 (2020) 075002, [1801.00797].
(55) A. Das, S. Jana, S. Mandal and S. Nandi, Probing right handed neutrinos at the LHeC and lepton colliders using fat jet signatures, Phys. Rev. D 99 (2019) 055030, [1811.04291].
(56) M. Chrzaszcz, M. Drewes, T. E. Gonzalo, J. Harz, S. Krishnamurthy and C. Weniger, A frequentist analysis of three right-handed neutrinos with GAMBIT, Eur. Phys. J. C 80 (2020) 569, [1908.02302].