This paper was converted on www.awesomepapers.org from LaTeX by an anonymous user.
Want to know more? Visit the Converter page.

\thankstext

e1e-mail: [email protected]

11institutetext: Abdus Salam School of Mathematical Sciences, Lahore, Pakistan

Topological Strings on Toric geometries in the presence of Lagrangian branes

M.Nouman Muteeb \thanksrefe1
(Received: date / Accepted: date )
Abstract

We propose expressions for refined open topological string partition function on certain non-compact Calabi Yau 3-folds with topological branes wrapped on the special lagrangian submanifolds. The corresponding web diagrams are partially compact and a lagrangian brane is inserted on one of the external legs. Partial compactification introduces a mass deformation in the corresponding gauge theory. We propose conjectures that equate these open topological string partition functions with the generating function of equivaraint indices on certain quiver moduli spaces. To obtain these conjectures we use the identification of topological string partition functions with equivariant indices on the instanton moduli spaces.

journal: Eur. Phys. J. C

1 Introduction

Topological strings Antoniadis:1993ze ; Neitzke:2004ni ; Assi:2014exa correspond to the subsector of full superstrings spectrum that is invariant under a non-trivial sub algebra of the extended supersymmetry algebra. The observables of this subsector describe various topological properties of the spacetime on which the observables are defined. In physical 4d effective field theory the topological strings with the so-called AtwistA-twist generate the the following F-term that depends on the vector multiplet moduli

Fg(𝒲2)g\displaystyle\int F_{g}(\mathcal{W}^{2})^{g} (1)

where 𝒲2\mathcal{W}^{2} is composed of the Weyl multiplet superfield 𝒲αβ\mathcal{W}_{\alpha\beta} and FgF_{g} is the genus gg topological string free energy. FgF_{g} for g2g\geq 2 describe coefficients in the scattering amplitude of 2g22g-2 graviphotons. The presence of D-branes imply consistent boundary conditions on the world sheet boundaries. In the topological sector the boundary conditions should preserve the BRST symmetry of the world sheet. From the target space (CY) perspective in the A-model the world sheet boundaries are mapped to a particular submanifold LL whose dimension is equal to the half that of CY Neitzke:2004ni and the restriction of the CY Kähler form ω\omega to LL vanishes. In general, open strings may end on LL resulting in the wrapping of an A-model brane on L.
The problem of computing the unrefined topological string amplitudes on the toric CY in the presence of both external as well as internal branes was solved by the technique of the topological vertex Aganagic:2003db . On the other hand the refined topological string amplitude in the presence of internal branes turns out to be a subtle problem. Certain surface operators in 4d gauge theory can be realised by wrapping D4-branes on the Lagrangian 3-cycles of the CY 3-fold with the other two directions extending along transverse 24\mathbb{R}^{2}\subset\mathbb{R}^{4}. These two transverse directions correspond to equivariant parameters ϵ1,ϵ2\epsilon_{1},\epsilon_{2} or t=eiϵ1,q=eiϵ2t=e^{-i\epsilon_{1}},q=e^{i\epsilon_{2}}. The topological branes can be put on the external non-compact legs or the internal compact legs of the toric diagram associated to the CY. Progress has been made in Kozcaz:2018ndf , where authors discuss refinement of holonomies for refined open topological string amplitudes. This was the case for topological branes on the external leg of the toric web diagram. In the presence of internal brane(s), there is a mismatch between the results as computed by using refinement of holonomy prescription and the geometric transition.
In this note we suggest that by utilising the equivalence of topological string amplitudes with equivariant indices Li:2004ef on framed moduli spaces, it may be possible to compute the refined open string amplitudes in the presence of internal branes. To this end, the authors Bruzzo:2010fk proposed a conjecture that equates the refined open string invariants of the special lagrangian branes in toric Calabi Yau 3-folds, with the Witten index of the supersymmetric quantum mechanical model describing the BPS states attached to the surface defect. The Witten index can be interpreted as the Euler characteristic of the moduli space described by the quantum mechanical model. In other words the conjecture Bruzzo:2010fk equates the generating function of refined open string invariants of the special lagrangian branes with the instanton partition function in the presence of surface defects. The conjecture was checked to be true to high orders in the asymptotic expansion. According to the geometric engineering argument the M5M5 branes wrapped on a submanifold of the CY give rise to effective five dimensional gauge theories with surface defect.
In the case of partially compactified toric web diagram the refined open topological string amplitude is equated to the generating function of χy\chi_{y} genus on the same moduli space Li:2004ef ; Hollowood:2003cv ; Chuang:2013wpa . The Kähler parameter of this new compact direction corresponds to a massive adjoint hyper in the 5d gauge theory Hollowood:2003gr . We generalise the results of Bruzzo:2010fk and state the conjectures in the case of partial compactification of the resolved conifold, O(1)O(1)1O(-1)\oplus O(-1)\to\mathbb{P}^{1} and the partial compactification of the total space of the canonical bundle 𝒪(2,2)\mathcal{O}(-2,-2) of P1×P1P^{1}\times P^{1}. In the fully compactified case we give the expression for the generating function of elliptic genus of the defect moduli space and speculate about its relevance for the refined open topological string amplitude in the presence of internal branes.
In section (2) we restate the conjecture proved in Bruzzo:2010fk , about the equivalence of partition function on quiver moduli space with open Gromov-Witten invariants of certain non-compact Calabi-Yau 3-folds. In section (3) we compute the generating function of χy\chi_{y} genus on the quiver moduli space. The expression we obtain is not a polynomial in the mass-deformation parameter yy. In the next section (3.3) we use analytic continuation to write the χy\chi_{y}-genus as a polynomial in yy. In section 3.4 we compute the generating function χy\chi_{y}-genus for quiver moduli space for rank r=2r=2, which corresponds to the refined open topological string partition function on the partially compactified web of the Hirzebruch surface F0F_{0}. In section (4) we briefly discuss the identification of the Donaldson-Thomas partition function of CY3-fold and K-theory partition function of 5d supersymmetric gauge theory motivated by the geometric engineering argument. This identification was obtained for the unrefined case by making a change of variables. Using certain consistency conditions we propose a generalisation of this change of variables to the refined case. In the last section (5) we given an expression for the elliptic genus on the same moduli space and suggest that it may be related to the open topological string partition function on non-compact Calabi-Yau 3-fold whose corresponding web diagram is fully compactified.The appendices (A,B,C) contain the refined topological vertex amplitudes for the remaining preferred directions and the appendix (D) contain a summary of the virtual equivariant localisation and fixed point theorems.

2 Quiver model and Open topological string amplitudes

A standard D-brane construction of 5d5d gauge theory with eight supercharges involves Bruzzo:2010fk D6D6-branes wrapped on the holomorphic curves in a non-compact K3K3 surface SS. The type of quiver is defined by the intersection matrix of the configuration of (2)(-2)-rational curves after suitable resolution. In the present context only ArA_{r}-type singularities are considered which are amenable to analysis with toric geometry techniques. So the type IIA vacuum we consider is given by S×S1×5S×C×4S\times S^{1}\times\mathbb{R}^{5}\equiv S\times C^{*}\times\mathbb{R}^{4}. The low energy limit of D6D6-branes wrapped on these rational curves gives rise to 5d5d quiver gauge theory with eight supercharges. In the more general type IIA superstrings setup we can add D4-branes wrapping the special Lagrangian submanifolds, along with D2 branes ending on these D4-branes. D2-branes are wrapped on (2)(-2)-rational curves. In other words D4-branes serve as defect operators and D2 branes correspond to the BPS states bound to the defect operators. The world volume of D4-brane is L×2L\times\mathbb{R}^{2}, where LT×CL\subset T\times C^{*} is the special Lagrangian submanifold and 24\mathbb{R}^{2}\subset\mathbb{R}^{4}. By construction a toric Calabi Yau 3-fold XX admits a symplectic U(1)3U(1)^{3} action and the resulting moment map ρ:X3\rho:X\to\mathbb{R}^{3} to the so-called Delzant polytope. The collection of U(1)3U(1)^{3} preserving compact and non-compact rational holomorphic curves of XX define its toric skeleton. The toric skeleton is mapped by ρ\rho to a trivalent graph Δ\Delta in 3\mathbb{R}^{3}. The special lagrangian submanifold LL under consideration is topologically equivalent to S1×2S^{1}\times\mathbb{R}^{2} and is mapped to a half real line which intersects a 1-face of the graph Δ\Delta. The external lagrangian cycles intersect the non-compact components of the skeleton, whereas the internal lagrangian cycles intersect compact components. In the web diagrams we always show the branes wrapped on the lagrangian cycles by dashed lines.
The D4-branes are wrapped on the Lagrangian cycles of the CY3-fold, with two directions extending along one of the 2s\mathbb{R}^{2}s of the transverse 4\mathbb{R}^{4}. For the unrefined case the choice of 2\mathbb{R}^{2} is immaterial. For the refined case the two 2\mathbb{R}^{2}s inside 4\mathbb{R}^{4} are rotated by q=eiϵ1,t=eiϵ2q=e^{i\epsilon_{1}},t=e^{-i\epsilon_{2}} corresponding to a particular choice of complex structure. Depending on which 2\mathbb{R}^{2} the D4-brane is extended along, it is called either a q-brane or a t-brane Kozcaz:2018ndf . In the presence of D4-branes the open topological string amplitudes are the generating functions of BPS degeneracies of D2 branes. These D2-branes are wrapped on smooth curves whose boundaries lie on D4-branes. The boundary conditions are necessary for the complete specification of the open string amplitudes and are given by gauge invariant combinations of holonomy operators.
The pure SU(r),r2SU(r),r\geq 2 gauge theories in 5d can be engineered by certain non-compact CY 3-folds. To construct these 3-folds , the total space of 𝒪(1)𝒪(1)1\mathcal{O}(-1)\oplus\mathcal{O}(-1)\to\mathbb{P}^{1} is orbifolded by the action (z1,z2)(e2πirz1,e2πirz2)(z_{1},z_{2})\to(e^{\frac{2\pi i}{r}}z_{1},e^{-\frac{2\pi i}{r}}z_{2}) on the fiber coordinates (z1,z2)(z_{1},z_{2}). The resulting space is singular and can be resolved to a smooth CY 3-fold which contains (r1)(r-1) geometrically ruled surfaces glued together. It is equivalent to the resolved Ar1A_{r-1} fibration over 1\mathbb{P}^{1}. The compact part of the geometry consist of r1r-1 Hirzebruch surfaces glued together and the normal geometry of a base 1\mathbb{P}^{1} in the p1p-1-th and pp-th Hirzebruch surfaces is 𝒪(r+2p2)𝒪(r2p)\mathcal{O}(-r+2p-2)\oplus\mathcal{O}(r-2p). Formally allowing the value r=1r=1 corresponds to the resolved conifold, the total space of 𝒪(1)𝒪(1)1\mathcal{O}(-1)\oplus\mathcal{O}(-1)\to\mathbb{P}^{1}.
Partially compactifying the web diagrams, which becomes non-planar, changes the CY3-fold to an elliptic CY3-fold which has the structure of the form 2/r×fT2\mathbb{C}^{2}/\mathbb{Z}_{r}\times_{f}T^{2}. M-theory compactification on this geometry engineers 5d5d 𝒩=1\mathcal{N}=1^{*} gauge theory with a single adjoint hypermultiplet Hollowood:2003cv ; Iqbal:2008ra .
The fields of quantum mechanical system, describing the BPS states bound to the surface operators, arise from low energy modes of D2D6D2-D6, D2D4D2-D4 and D2D2D2-D2 configuration of D-branes in type IIA strings. The field content is summarised as

D2D6\displaystyle D2-D6 :\displaystyle: two(0,2)chiralmultiplets\displaystyle two\quad(0,2)\quad chiral\quad multiplets
D2D4\displaystyle D2-D4 :\displaystyle: a(0,2)chiralmultipletanda(0,2)vector\displaystyle a\quad(0,2)\quad chiral\quad multiplet\quad and\quad a\quad(0,2)\quad vector\quad
. multiplet\displaystyle multiplet
D2D2\displaystyle D2-D2^{\prime} :\displaystyle: two(0,2)chiralmultipletsandtwo(0,2)\displaystyle two\quad(0,2)\quad chiral\quad multiplets\quad and\quad two\quad(0,2)\quad
. Fermimultiplets\displaystyle Fermi\quad multiplets
D2D6\displaystyle D2^{\prime}-D6 :\displaystyle: asingle(0,2)Fermimultiplet.\displaystyle a\quad single\quad(0,2)\quad Fermi\quad multiplet. (2)

An intricate analysis Bruzzo:2010fk shows that the moduli space of the supersymmetric vacua of the quantum mechanical model is isomorphic to the data defining ADHM type quiver. In a certain region of the moduli space it is interpreted in terms of certain generalised vector bundles.
The D2-brane effective action is derived by the dimensional reduction of the field contents of quiver diagram (1). The quiver diagram describes a vector space of supersymmetric flat directions parametrised by the fields (A1,A2,I,J,B2,f,g,σ1,σ2)(A_{1},A_{2},I,J,B_{2},f,g,\sigma_{1},\sigma_{2}) as follows

End (V1)2Hom(W,V1)Hom(V1,W)End(V2)Hom(V1,V2)Hom(V2,V1)\displaystyle(V_{1})^{\oplus 2}\oplus\mbox{Hom}(W,V_{1})\oplus\mbox{Hom}(V_{1},W)\oplus\mbox{End}(V_{2})\oplus\mbox{Hom}(V_{1},V_{2})\oplus\mbox{Hom}(V_{2},V_{1})
\displaystyle\oplus u(V1)u(V2)\displaystyle\mbox{u}(V_{1})\oplus\mbox{u}(V_{2})

for hermitian inner product vector spaces V1,V2V_{1},V_{2} and WW. The vacuum equations of the D2-branes effective action as given below define a moduli space

[A1,A1]+[A2,A2]+IIJJ+ffgg=ζ1,[B2,B2]ffgg=ζ2,\displaystyle[A_{1},A_{1}^{\dagger}]+[A_{2},A_{2}^{\dagger}]+II^{\dagger}-J^{\dagger}J+ff^{\dagger}-g^{\dagger}g=\zeta_{1},\qquad[B_{2},B_{2}^{\dagger}]-f^{\dagger}f-gg^{\dagger}=\zeta_{2},
gA1=0,A1f=0,gI=0,Jf=0[A1,A2]+IJ=0A2ffB2=0,\displaystyle gA_{1}=0,\quad A_{1}f=0,\quad gI=0,\quad Jf=0\quad[A_{1},A_{2}]+IJ=0A_{2}f-fB_{2}=0,
gA2B2g=0,fg=0,\displaystyle gA_{2}-B_{2}g=0,fg=0,
[σ1,A1]=0,[σ1,A2]=0,[σ2,B2]=0,σ1I=0,Jσ1=0,σ1ffσ2=0,\displaystyle[\sigma_{1},A_{1}]=0,\quad[\sigma_{1},A_{2}]=0,\quad[\sigma_{2},B_{2}]=0,\quad\sigma_{1}I=0,\quad J\sigma_{1}=0,\quad\sigma_{1}f-f\sigma_{2}=0,
gσ1σ2g=0.\displaystyle g\sigma_{1}-\sigma_{2}g=0.

The moduli space parametrize U(V1)×U(V2)\mbox{U}(V_{1})\times\mbox{U}(V_{2}) gauge inequivalent solutions to (2). An important result proven in Bruzzo:2010fk shows that the moduli space defined by the last set of equations is isomorphic to a different moduli space for generic values of Fayet Illuopolous parameters.The later moduli space comprises of stable representations of the enhanced ADHM quiver, also described in terms of framed torsion free sheaves on the projective plane. The virtual smoothness in our context means that moduli space of stable representations of the ADHM quiver can be embedded in a smooth variety which is a hyper kähler quotient.

V2{V_{2}}V1{V_{1}}W Λ  {W{}{{}}{}{{}{}{}{{}}{{{{}{}{}{}}}{{{}{}{}{}}}{{}{}{}{}}}}{}{}{}{}{}{}\pgfsys@beginscope\pgfsys@setlinewidth{0.8pt}{}{{{\pgfsys@beginscope\pgfsys@setdash{}{0.0pt}\pgfsys@roundcap\pgfsys@roundjoin{}{}{}{}{}{}{}\pgfsys@moveto{-2.56pt}{3.12257pt}\pgfsys@curveto{-2.0923pt}{1.24901pt}{-1.05006pt}{0.3643pt}{0.0pt}{0.0pt}\pgfsys@curveto{-1.05006pt}{-0.3643pt}{-2.0923pt}{-1.24901pt}{-2.56pt}{-3.12257pt}\pgfsys@stroke\pgfsys@endscope}}}{}{}{{}}\pgfsys@moveto{0.0pt}{0.0pt}\pgfsys@curveto{0.0pt}{25.14285pt}{-20.38173pt}{45.52458pt}{-45.52458pt}{45.52458pt}\pgfsys@curveto{-61.78911pt}{45.52458pt}{-76.81787pt}{36.84769pt}{-84.95013pt}{22.76228pt}\pgfsys@curveto{-89.35004pt}{15.14128pt}{-91.44717pt}{6.40536pt}{-91.02849pt}{-1.58376pt}\pgfsys@stroke\pgfsys@invoke{}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@transformcm{0.05232}{-0.99863}{0.99863}{0.05232}{-91.00757pt}{-1.9832pt}\pgfsys@invoke{\lxSVG@closescope}\pgfsys@invoke{\lxSVG@closescope}\pgfsys@endscope}}{{}}}}\pgfsys@invoke{\lxSVG@closescope}\pgfsys@endscope{{}}\hbox{\hbox{{\pgfsys@beginscope{{}{}{{ {}{}}}{ {}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-12.56628pt}{25.95276pt}\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\hbox{$${$\Lambda_{-}$}$$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}}B2+\scriptstyle{B_{2+}}Φ+Ω\scriptstyle{\Phi_{+}\Omega_{-}}Γ+Ψ\scriptstyle{\Gamma_{+}\Psi_{-}}A1+A2+\scriptstyle{A_{1+}A_{2+}}χ\scriptstyle{\chi_{-}}\scriptstyle{\mathcal{I}}𝒥\scriptstyle{\mathcal{J}}
Figure 1: quiver diagram encoding D2-branes effective action

The enhanced ADHM quiver is given in figure (3) along with the relationsrelations

α!α2α2α1+ξη,α1ϕ,α2ϕϕβ,ηϕ,γξ,ϕγ,γα1,γα2βγ.\displaystyle\alpha_{!}\alpha_{2}-\alpha_{2}\alpha_{1}+\xi\eta,\quad\alpha_{1}\phi,\quad\alpha_{2}\phi-\phi\beta,\quad\eta\phi,\quad\gamma\xi\quad,\phi\gamma,\gamma\alpha_{1},\gamma\alpha_{2}-\beta\gamma. (5)

A triple of vector spaces (V1,V2,W)(V_{1},V_{2},W) is assigned to the vertices (e1,e2,e)(e_{1},e_{2},e_{\infty}) as a representation. Similarly the linear maps (A1,A2,I,J,B,f,g)(A_{1},A_{2},I,J,B,f,g) represent the arrows (α1,α2,ξ,η,β,ϕ,γ)(\alpha_{1},\alpha_{2},\xi,\eta,\beta,\phi,\gamma). The resulting quiver representation is moded out by the relations (5). If we define (r,n1,n2):=(dim(W),dim(V1),dim(V2))(r,n_{1},n_{2}):=(\mbox{dim}(W),\mbox{dim}(V_{1}),\mbox{dim}(V_{2})), then this triple of positive integers define the numerical type of the quiver. The framing of the enhanced quiver representation corresponds to the existence of an isomorphism h:Wr\mbox{h}:\mbox{W}\to\mathbb{C}^{r}. Moreover a stable111more precisely θ\theta-semistable representation of type (r,n1,n2)(r,n_{1},n_{2}) implies the existence of a set of parameters θ=(θ1,θ2,θ)\theta=(\theta_{1},\theta_{2},\theta_{\infty}) satisfying the relation n1θ1+n2θ2+rθn_{1}\theta_{1}+n_{2}\theta_{2}+r\theta_{\infty} such that

  • Any subrepresentaion of type (0,m1,m2)(0,m_{1},m_{2}) satisfies m1θ1+m2θ20m_{1}\theta_{1}+m_{2}\theta_{2}\leq 0.

  • Any subrepresentaion of type (r,m1,m2)(r,m_{1},m_{2}) satisfies m1θ1+m2θ2+rθ0m_{1}\theta_{1}+m_{2}\theta_{2}+r\theta_{\infty}\leq 0

The following result gives criterion for generic stability conditions:
Given a quiver representation \mathcal{R} of type (r,n1,n2)(r,n_{1},n_{2}) and theta parameters satisfying θ2>0,θ1+n2θ2<0\theta_{2}>0,\theta_{1}+n_{2}\theta_{2}<0, then the following three statements are equivalent

  • 1. \mathcal{R} is θ\theta-semistable

  • 2.\mathcal{R} is θ\theta-stable

  • 3. (a) f:V2V1f:V_{2}\to V_{1} is injective and g:V1V2g:V_{1}\to V_{2} is identically zero
    (b) The data 𝒜=(V1,W,A1,A2,I,J)\mathcal{A}=(V_{1},W,A_{1},A_{2},I,J) satisfies the ADHM stability conditions.

Consider the vector spaces V1,V2,WV_{1},V_{2},W with positive definite dimensions n1,n2,rn_{1},n_{2},r and the direct sum of vector spaces

X(r,n1,n2)=End(V1)2Hom(W,V1)Hom(V1,W)End(V2)Hom(V1,V2)Hom(V2,V1)\displaystyle X(r,n_{1},n_{2})=\mbox{End}(V_{1})^{\oplus 2}\oplus\mbox{Hom}(W,V_{1})\oplus\mbox{Hom}(V_{1},W)\oplus\mbox{End}(V_{2})\oplus\mbox{Hom}(V_{1},V_{2})\oplus\mbox{Hom}(V_{2},V_{1})

X(r,n1,n2)X(r,n_{1},n_{2}) admits a GL(V1)×GL(V2)\mbox{GL}(V_{1})\times\mbox{GL}(V_{2}) action defined as

(g1,g2)×(A1,A2,I,J,B2,f,g)(g1A1g11,g1A2g11,Jg11,g1I,g2B2g21,g1fg21,g2gg11)\displaystyle(g_{1},g_{2})\times(A_{1},A_{2},I,J,B_{2},f,g)\to(g_{1}A_{1}g_{1}^{-1},g_{1}A_{2}g_{1}^{-1},Jg_{1}^{-1},g_{1}I,g_{2}B_{2}g_{2}^{-1},g_{1}fg_{2}^{-1},g_{2}gg_{1}^{-1})

It is clear that 𝒢=GL(V1)×GL(V2)\mathcal{G}=\mbox{GL}(V_{1})\times\mbox{GL}(V_{2}) preserves the subset X0(r,n1,n2)X(r,n1,n2)X_{0}(r,n_{1},n_{2})\in X(r,n_{1},n_{2}) defined by (2). The space X0(r,n1,n2)X_{0}(r,n_{1},n_{2}) parametrises the representations =(V1,V2,W,A1,I,J,B2,f,g)\mathcal{R}=(V_{1},V_{2},W,A_{1},I,J,B_{2},f,g) with two framed representations 1,2\mathcal{R}_{1},\mathcal{R}_{2} being equivalent if the corresponding points in X0(r,n1,n2)X_{0}(r,n_{1},n_{2}) belong to the same GL(V1)×GL(V2)GL(V_{1})\times GL(V_{2}) orbit. The space X0(r,n1,n2)X_{0}(r,n_{1},n_{2}) can be projectivized to a scheme as follows

𝒩θss(r,n1,n2)=X0(r,n1,n2)//χ𝒢:=Proj(n0A(X0(r,n1,n2))G,χn)\displaystyle\mathcal{N}_{\theta}^{ss}(r,n_{1},n_{2})=X_{0}(r,n_{1},n_{2})//_{\chi}\mathcal{G}:=Proj\bigg{(}\oplus_{n\geq 0}A(X_{0}(r,n_{1},n_{2}))^{G,\chi^{n}}\bigg{)} (8)

with the notation A(X0(r,n1,n2))𝒢,χn:={fA(X0(r,n1,n2))|f(g.x)=χ(g)nf(x)gG}A(X_{0}(r,n_{1},n_{2}))^{\mathcal{G},\chi^{n}}:=\{f\in A(X_{0}(r,n_{1},n_{2}))|f(g.x)=\chi(g)^{n}f(x)\forall g\in G\}. There exists an open subscheme 𝒩θs(r,n1,n2)\mathcal{N}_{\theta}^{s}(r,n_{1},n_{2}) of 𝒩θss(r,n1,n2)\mathcal{N}_{\theta}^{ss}(r,n_{1},n_{2}) whose 𝒢\mathcal{G}-orbits are χ\chi-stable222The character function χ:𝒢×\chi:\mathcal{G}\to\mathbb{C}^{\times}, where 𝒢\mathcal{G} acts on the space X(r,n1,n2)X(r,n_{1},n_{2}), furnishes a definition of χ\chi-stability. For this consider the existence of a polynomial q(x)q(x) on X(r,n1,n2)X(r,n_{1},n_{2}) such that q((g1,g2).x)=χ(g1,g2)nq(x)q((g_{1},g_{2}).x)=\chi(g_{1},g_{2})^{n}q(x) for positive definite integer nn. If q(x0)0q(x_{0})\neq 0, x0x_{0} is called χ\chi-semistable. Moreover if Δ𝒢\Delta\subset\mathcal{G} acts trivially on X(r,n1,n2)X(r,n_{1},n_{2}) such that dim(𝒢.x0)=dim(𝒢/Δ)dim(\mathcal{G}.x_{0})=dim(\mathcal{G}/\Delta) and the action of 𝒢\mathcal{G} on all such x0x_{0} is closed, then then x0x_{0} is called χ\chi-stable.. In the parameter space defined by the inequalities θ2>0,θ1+n2θ2<0\theta_{2}>0,\theta_{1}+n_{2}\theta_{2}<0 the framed representations of the enhanced quiver that satisfy condition (3)(3) can simply be denoted by 𝒩(r,n1,n2)\mathcal{N}(r,n_{1},n_{2}) by dropping subscripts and superscripts.
The matter couplings in the quantum mechanical model corresponds to the three tautological bundles 𝒱1,𝒱2,𝒲\mathcal{V}_{1},\mathcal{V}_{2},\mathcal{W} on the moduli space 𝒩(r,n1,n2)\mathcal{N}(r,n_{1},n_{2}). These bunldes are defined by 𝒲=𝒪𝒩(r,n1,n2)r\mathcal{W}=\mathcal{O}^{\oplus r}_{\mathcal{N}(r,n_{1},n_{2})}, 1=det(𝒱1)\mathcal{L}_{1}=\mbox{det}(\mathcal{V}_{1}) and 2=det(𝒱2)\mathcal{L}_{2}=\mbox{det}(\mathcal{V}_{2}). Moreover several copies of 1,2\mathcal{L}_{1},\mathcal{L}_{2} can be tensored to give rise to the mixed line bundles p1,p2=1p12p2\mathcal{L}_{p_{1},p_{2}}=\mathcal{L}_{1}^{\otimes p_{1}}\otimes\mathcal{L}_{2}^{\otimes p_{2}}.
The existence of the morphism s:𝒩(r,n1,n2)(r,n1n2)s:\mathcal{N}(r,n_{1},n_{2})\to\mathcal{M}(r,n_{1}-n_{2}), where (r,n1n2)\mathcal{M}(r,n_{1}-n_{2}) is the moduli space of ADHM data of type (r,n)(r,n), plays a simplifying role in the application of the equivariant fixed point theorems. The tangent space at a point of 𝒩(r,n1,n2)\mathcal{N}(r,n_{1},n_{2}) is isomorphic to the difference H1(𝒞())H2(𝒞())H^{1}(\mathcal{C}(\mathcal{R}))-H^{2}(\mathcal{C}(\mathcal{R})) for =(A1,A2,I,J,B2,f)\mathcal{R}=(A_{1},A_{2},I,J,B_{2},f) representing a point of 𝒩(r,n1,n2)\mathcal{N}(r,n_{1},n_{2}), where the complex 𝒞()\mathcal{C}(\mathcal{R}) is defined by

End(V1)End(V2){\mbox{End}(V_{1})\oplus\mbox{End}(V_{2})}End(V1)2Hom(W,V1)Hom(V1,W)End(V2)Hom(V2,V1){\mbox{End}(V_{1})^{\oplus 2}\oplus\mbox{Hom}(W,V_{1})\oplus\mbox{Hom}(V_{1},W)\oplus\mbox{End}(V_{2})\oplus\mbox{Hom}(V_{2},V_{1})}d0\scriptstyle{d_{0}}
End(V1)Hom(V2,V1)2Hom(V2,W){\mbox{End}(V_{1})\oplus\mbox{Hom}(V_{2},V_{1})^{\oplus 2}\oplus\mbox{Hom}(V_{2},W)}d1\scriptstyle{d_{1}}
Hom(V2,V1){\mbox{Hom}(V_{2},V_{1})}d2\scriptstyle{d_{2}}
Figure 2: 𝒞()\mathcal{C}(\mathcal{R})

with the differentials defined as

d0(α1,α2)t=([α1,A1],[α2,A2],α1,Jα1,[α2,B2],α1ffα2)t\displaystyle d_{0}(\alpha_{1},\alpha_{2})^{t}=([\alpha_{1},A_{1}],[\alpha_{2},A_{2}],\alpha_{1},-J\alpha_{1},[\alpha_{2},B_{2}],\alpha_{1}f-f\alpha_{2})^{t}
d1(a1,a2,i,j,b2,ϕ)t=\displaystyle d_{1}(a_{1},a_{2},i,j,b_{2},\phi)^{t}=
([a1,A2]+[A1,a2]+Ij+iJ,A1ϕ+a1f,A2ϕ+a2ffb2ϕB2,jf+jϕ)t\displaystyle([a_{1},A_{2}]+[A_{1},a_{2}]+Ij+iJ,A_{1}\phi+a_{1}f,A_{2}\phi+a_{2}f-fb_{2}-\phi B_{2},jf+j\phi)^{t}
d2(c1,c2,c3,c4)t=c1f+A2c2c2B2A1c3Ic4\displaystyle d_{2}(c_{1},c_{2},c_{3},c_{4})^{t}=c_{1}f+A_{2}c_{2}-c_{2}B_{2}-A_{1}c_{3}-Ic_{4} (9)

Note that H1(𝒞())H^{1}(\mathcal{C}(\mathcal{R})) parametrises the infinitesimal deformations of \mathcal{R} and H2(𝒞())H^{2}(\mathcal{C}(\mathcal{R})) the obstructions to the deformations. Moreover the conditions H0(𝒞())=H3(𝒞())=0H^{0}(\mathcal{C}(\mathcal{R}))=H^{3}(\mathcal{C}(\mathcal{R}))=0 imply the stability of a framed representation. The equivariant virtual Euler characteristic of this determinant line bundle is arranged into a partition function of the quantum mechanical model.

e2{e_{2}}e1{e_{1}}e{e_{\infty}}B2+\scriptstyle{B_{2+}}ϕ\scriptstyle{\phi}γ\scriptstyle{\gamma}α1\scriptstyle{\alpha_{1}}α2\scriptstyle{\alpha_{2}}η\scriptstyle{\eta}ξ\scriptstyle{\xi}
Figure 3: enhanced ADHM quiver

The application of virtual equivariant localization requires the determination of fixed points of the torus action on the moduli space of the stable representations of the nested ADHM quivers. The moduli space, 𝒩(r,n+d,d)\mathcal{N}(r,n+d,d), for which we compute the Euler characteristic, the χy\chi_{y} genus and the elliptic genus is described by the stable representations of an enhanced ADHM quiver of type (n+d,d,r)(n+d,d,r). Note that the parameter rr is the rank of the 5d gauge group and (n+d,d)(n+d,d) are related to the number of D2,D2D2,D2^{\prime} branes. The BPS counting function is the Witten index of the supersymmetric quantum mechanics. The supersymmetric ground states are in one to one correspondence with cohomology classes in iH0,i(𝒩(r,n1,n2))\oplus_{i}H^{0,i}(\mathcal{N}(r,n_{1},n_{2})). Note that in the limit of decoupling the surface operator 𝒩(r,n+d,d)\mathcal{N}(r,n+d,d) collapses to (r,n)\mathcal{M}(r,n) where (r,n)\mathcal{M}(r,n) is the suitably compactified and non-singular moduli space of instantons on 2\mathbb{C}^{2}. The later moduli space is isomorphic to the moduli space of rank NN torsion free sheaves with second Chern class kk on 2\mathbb{P}^{2}. So there exists a morphism q:𝒩(r,n+d,d)(r,n)q:\mathcal{N}(r,n+d,d)\to\mathcal{M}(r,n). This morphism is equivariant with respect to the torus action. This makes it possible to classify the torus fixed loci in 𝒩(r,n+d,d)\mathcal{N}(r,n+d,d).
In the following (μ,ν)(\mu,\nu) is a pair of nested Young diagrams satisfying the properties

  • νμ\nu\subseteq\mu

  • if (i,j)μν,then(i+1,j)μ(i,j)\in\mu\setminus\nu,then\quad(i+1,j)\notin\mu

Similarly if we denote an ordered sequence of Young diagrams by μ¯={μ𝟏,,μ𝐫}\bf{\underline{\mu}}=\{\mu^{1},...,\mu^{r}\} and ν¯={ν𝟏,,ν𝐫}\bf{\underline{\nu}}=\{\nu^{1},...,\nu^{r}\} then it is a nested sequence if it is pairwise (μa,νa)(\mu^{a},\nu^{a}) nested. The numerical type of this sequence is given by (|μ|¯,|ν¯|)(|\underline{\mu|},|\underline{\nu}|). For nested sequence the defining algebraic inequalities are described as

0caea1,0μiaνiaνi1aνia\displaystyle 0\leq c^{a}-e^{a}\leq 1,\qquad 0\leq\mu_{i}^{a}-\nu_{i}^{a}\leq\nu_{i-1}^{a}-\nu_{i}^{a} (10)

where ca,eac^{a},e^{a} denote the number of columns of μa\mu^{a} and νa\nu^{a} respectively and a=1,,ra=1,...,r and i0i\geq 0. The tangent space to the moduli space at a fixed point (μa,νa)(\mu^{a},\nu^{a}) of the torus 𝐓=××××(×)𝐫\bf{T}=\mathbb{C}^{\times}\times\mathbb{C}^{\times}\times(\mathbb{C}^{\times})^{r} is regarded as an element of the representation ring of the torus action and is given by the expression

Tμ¯,ν¯𝒩(r,n+d,d)\displaystyle T_{\underline{\mu},\underline{\nu}}\mathcal{N}(r,n+d,d) =\displaystyle= Tν¯(r,n)\displaystyle T_{\underline{\nu}}\mathcal{M}(r,n)
+\displaystyle+ a,b=1ri=2ea+1j=1μjbνjbRa1RbQ1ij(Q2μiaνjbs+1Q2νi1bνjbs+1)\displaystyle\sum_{a,b=1}^{r}\sum_{i=2}^{e^{a}+1}\sum_{j=1}^{\mu_{j}^{b}-\nu_{j}^{b}}R_{a}^{-1}R_{b}Q_{1}^{i-j}(Q_{2}^{\mu_{i}^{a}-\nu_{j}^{b}-s+1}-Q_{2}^{\nu^{b}_{i-1}-\nu_{j}^{b}-s+1})
+\displaystyle+ a,b=1rj=1cbs=1μjbνjbRa1RbQ1j+1Q2μ1aνjbs+1\displaystyle\sum_{a,b=1}^{r}\sum_{j=1}^{c^{b}}\sum_{s=1}^{\mu_{j}^{b}-\nu_{j}^{b}}R_{a}^{-1}R_{b}Q_{1}^{-j+1}Q_{2}^{\mu_{1}^{a}-\nu_{j}^{b}-s+1}
=\displaystyle= a,b=1rRa1Rb((i,j)νaQ1i(νb)jtQ2νiaj+1+(i,j)νbQ1(νa)jti+1Q2jνib)\displaystyle\sum_{a,b=1}^{r}R_{a}^{-1}R_{b}\big{(}\sum_{(i,j)\in\nu^{a}}Q_{1}^{i-(\nu^{b})^{t}_{j}}Q_{2}^{\nu_{i}^{a}-j+1}+\sum_{(i,j)\in\nu^{b}}Q_{1}^{(\nu^{a})^{t}_{j}-i+1}Q_{2}^{j-\nu_{i}^{b}}\big{)}
+\displaystyle+ a,b=1ri=2ea+1j=1μjbνjbRa1RbQ1ij(Q2μiaνjbs+1Q2νi1bνjbs+1)\displaystyle\sum_{a,b=1}^{r}\sum_{i=2}^{e^{a}+1}\sum_{j=1}^{\mu_{j}^{b}-\nu_{j}^{b}}R_{a}^{-1}R_{b}Q_{1}^{i-j}(Q_{2}^{\mu_{i}^{a}-\nu_{j}^{b}-s+1}-Q_{2}^{\nu^{b}_{i-1}-\nu_{j}^{b}-s+1})
+\displaystyle+ a,b=1rj=1cbs=1μjbνjbRa1RbQ1j+1Q2μ1aνjbs+1\displaystyle\sum_{a,b=1}^{r}\sum_{j=1}^{c^{b}}\sum_{s=1}^{\mu_{j}^{b}-\nu_{j}^{b}}R_{a}^{-1}R_{b}Q_{1}^{-j+1}Q_{2}^{\mu_{1}^{a}-\nu_{j}^{b}-s+1}

where R1,,Rr,Q1,Q2R_{1},...,R_{r},Q_{1},Q_{2} denote the one dimensional representations of 𝐓\bf{T} with their characters represented by ρ1,,ρr,q1,q2\rho_{1},...,\rho_{r},q_{1},q_{2} respectively.
According to this formula the fixed point locus is a finite set of points in one to one correspondence with pairs of nested sequences (μ¯,ν¯)=(μa,νa)1ar(\underline{\mu},\underline{\nu})=(\mu^{a},\nu^{a})_{1\leq a\leq r} of length rr. The type of Young diagrams is (|μ¯|,|ν¯|)=(n+d,n)(|\underline{\mu}|,|\underline{\nu}|)=(n+d,n).

2.1 Holomorphic Euler characteristic, χy\chi_{y} genus and elliptic genus

Consider Fantechi_2010 ; Bonelli:2019het a rank dd vector bundle EE on some moduli space XX. We can form the formal sums of the symmetric product StES_{t}E and the antisymmetric product ΛtE\Lambda_{t}E as

ΛtE=i=0d[ΛiE]ti,StE=i=0[SiE]ti\displaystyle\Lambda_{t}E=\sum_{i=0}^{d}[\Lambda^{i}E]t^{i},\quad S_{t}E=\sum_{i=0}[S^{i}E]t^{i} (12)

The moduli space XX admits a virtual cotangent bundle ΩX=(TX)\Omega_{X}=(T_{X})^{\vee} and the bundle of n-forms ΩXn=ΛnΩX\Omega_{X}^{n}=\Lambda^{n}\Omega_{X}. On the scheme XX one can consider the perfect333perfect because the complex EE^{\bullet} has local isomorphism with a complex of vector bundles that is finite Fantechi_2010 . obstruction theory EE^{\bullet} resolved to a complex of vector bundles [E1E0][E^{-1}\to E^{0}] and a virtual structure sheaf denoted by 𝒪Xvir\mathcal{O}_{X}^{vir}.The virtual tangent bundle TXvirT^{vir}_{X} is defined by the class [E0][E1][E_{0}]-[E_{1}], where the complex [E0E1][E_{0}\to E_{1}] is dual to the complex [E1E0][E^{-1}\to E^{0}]. The difference rank(E)rank(E1)\mbox{rank}(E_{)}-\mbox{rank}(E_{1}) defines what is called the virtual dimension of XX. For a vector bundle VV given on XX and [X]vir[X]^{vir} the virtual fundamental class of XX as an element of the (nm)(n-m)-th Chow group of XX with rational coefficients, the virtual holomorphic Euler characteristic is defined by

χvir(X,V)=χ(X,VOXvir)=[X]virch(V).td(TXvir)\displaystyle\chi^{vir}(X,V)=\chi(X,V\otimes O_{X}^{vir})=\int_{[X]^{vir}}\mbox{ch}(V).\mbox{td}(T_{X}^{vir}) (13)

Then under suitable conditions using the Riemann-Roch theorem the Hirzebruch χy\chi_{y} genus is expressed as. See the appendix (D) for a summary of the virtual localization.

χy(X,V)=[X]ch(ΛyTX)ch(V)td(TX)=[X](l=1devk)i=1nxi1yexi1exij=1m1euiuj(1yeui)\displaystyle\chi_{-y}(X,V)=\int_{[X]}\mbox{ch}(\Lambda_{-y}T_{X})\mbox{ch}(V)\mbox{td}(T_{X})=\int_{[X]}(\sum_{l=1}^{d}e^{v_{k}})\prod_{i=1}^{n}x_{i}\frac{1-ye^{-x_{i}}}{1-e^{-x_{i}}}\prod_{j=1}^{m}\frac{1-e^{-u_{i}}}{u_{j}(1-ye^{-u_{i}})}

where x1,,xnx_{1},...,x_{n} denotes the Chern roots of E0E_{0}, u1,,umu_{1},...,u_{m} denote the Chern roots of E1E_{1} and v1,..,vrv_{1},..,v_{r} denote the Chern roots of the vector bundle VV. Note that for y=0y=0, χyvir(X,V)\chi_{-y}^{vir}(X,V) reduces to the Euler characteristic

χ(X,V)=[X](l=1devk)i=1nxi1exij=1m1euiuj\displaystyle\chi(X,V)=\int_{[X]}(\sum_{l=1}^{d}e^{v_{k}})\prod_{i=1}^{n}\frac{x_{i}}{1-e^{-x_{i}}}\prod_{j=1}^{m}\frac{1-e^{-u_{i}}}{u_{j}}

The moduli space 𝒩(r,n+d,d)\mathcal{N}(r,n+d,d) is in general non-compact and the cohomology groups H0,iH^{0,i} are not well defined. However due to the toric 𝐓=××××(×)𝐫\bf{T}=\mathbb{C}^{\times}\times\mathbb{C}^{\times}\times(\mathbb{C}^{\times})^{r} action on the moduli space, the Atiyah-Singer fixed point theorems can be applied to compute equivariant Euler character. The equivariant Euler character is an element of the quotient field of the representation ring of 𝐓\bf{T} and hence makes sense in the presence of non-compactness. The Euler character or the quiver partition function χT(𝒩(r,n+d,d))\chi_{T}(\mathcal{N}(r,n+d,d)) can be generalised by coupling the quiver quantum mechanical system with the line bundles (p1,p2)\mathcal{L}_{(p_{1},p_{2})} parametrised by two integers (p1,p2)2(p_{1},p_{2})\in\mathbb{Z}^{2}. This coupling can be interpreted as the Chern Simons terms in the M-theory lift of the type IIA configuration. A generating function can thus be composed

Z4d,quiver(q1,q2,ρa,Q)=n0chTχT(𝒩(r,n+d,d),S(p1,p2))\displaystyle Z^{4d,quiver}(q_{1},q_{2},\rho_{a},Q)=\sum_{n\geq 0}\mbox{ch}_{T}\chi_{T}(\mathcal{N}(r,n+d,d),S\otimes\mathcal{L}_{(p_{1},p_{2})}) (16)

where (q1,q2,ρ1,,ρr)(q_{1},q_{2},\rho_{1},...,\rho_{r}) denote the characters of the generators (Q1,Q2,R1,,Rr,Q)(Q_{1},Q_{2},R_{1},...,R_{r},Q) as defined before.
Finally the generating function for the equivariant Euler characteristics can be written as follows

Z4d,quiver(q1,q2,ρa,Q)\displaystyle Z^{4d,quiver}(q_{1},q_{2},\rho_{a},Q) =\displaystyle= kQkχT(k,q1,q2,ρa))\displaystyle\sum_{k}Q^{k}\chi_{T}(k,q_{1},q_{2},\rho_{a})) (17)
=\displaystyle= kQk|ν|=k\displaystyle\sum_{k}Q^{k}\sum_{|\nu|=k}
×\displaystyle\times 1(i,j)νa(1q1(νb)jtq2jνia1)(i,j)νb(1q1(νa)jtq2jνibj)\displaystyle\frac{1}{\prod_{(i,j)\in\nu^{a}}(1-q_{1}^{(\nu^{b})^{t}_{j}}q_{2}^{j-\nu_{i}^{a}-1})\prod_{(i,j)\in\nu^{b}}(1-q_{1}^{(\nu^{a})^{t}_{j}}q_{2}^{j-\nu_{i}^{b}-j})}
×\displaystyle\times (μ,ν)|μ|=|ν|+d[i=1cas=1μiaνiaρ1qa1iq2νias+1\displaystyle\sum_{\begin{subarray}{c}(\mu,\nu)\\ |\mu|=|\nu|+d\end{subarray}}\Bigg{[}\prod_{i=1}^{c^{a}}\prod_{s=1}^{\mu^{a}_{i}-\nu^{a}_{i}}\rho_{1}q_{a}^{1-i}q_{2}^{-\nu^{a}_{i}-s+1}
×\displaystyle\times 1i=2ea+1j=1cbs=1μjbνjb(1q1jiq2νjb+sμia1)\displaystyle\frac{1}{\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu_{j}^{b}}(1-q_{1}^{j-i}q_{2}^{\nu_{j}^{b}+s-\mu_{i}^{a}-1})}
×\displaystyle\times i=2ea+1j=1cbs=1μjbνjb(1q1jiq2νjb+sνi1a1)\displaystyle\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu_{j}^{b}}(1-q_{1}^{j-i}q_{2}^{\nu_{j}^{b}+s-\nu_{i-1}^{a}-1})
×\displaystyle\times 1j=1cbs=1μjbνjb(1q1j1q2νjb+sμ1a1)]\displaystyle\frac{1}{\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu^{b}_{j}}(1-q_{1}^{j-1}q_{2}^{-\nu^{b}_{j}+s-\mu_{1}^{a}-1})}\Bigg{]}
\displaystyle\equiv kQk|ν|=k\displaystyle\sum_{k}Q^{k}\sum_{|\nu|=k}
×\displaystyle\times 1(i,j)νa(1q1(νb)jtq2jνia1)(i,j)νb(1q1(νa)jtq2jνibj)\displaystyle\frac{1}{\prod_{(i,j)\in\nu^{a}}(1-q_{1}^{(\nu^{b})^{t}_{j}}q_{2}^{j-\nu_{i}^{a}-1})\prod_{(i,j)\in\nu^{b}}(1-q_{1}^{(\nu^{a})^{t}_{j}}q_{2}^{j-\nu_{i}^{b}-j})}
×\displaystyle\times (μ,ν)|μ|=|ν|+dW(μ,ν)(q1,q2,ρa,y)\displaystyle\sum_{\begin{subarray}{c}(\mu,\nu)\\ |\mu|=|\nu|+d\end{subarray}}W_{(\mu,\nu)}(q_{1},q_{2},\rho_{a},y)

Here Wν,d(q1,q2,ρa,y)(μ,ν)|μ|=|ν|+dW(μ,ν)(q1,q2,ρa,y)W_{\nu,d}(q_{1},q_{2},\rho_{a},y)\equiv\sum_{\begin{subarray}{c}(\mu,\nu)\\ |\mu|=|\nu|+d\end{subarray}}W_{(\mu,\nu)}(q_{1},q_{2},\rho_{a},y) contains the information about the surface defect and the BPS states bound to it.
In Bruzzo:2010fk the refined open topological string partition function on the resolved conifold .i.e. the total space of 𝒪(1)𝒪(1)1\mathcal{O}(-1)\oplus\mathcal{O}(-1)\to\mathbb{P}^{1}, and the total space of the canonical bundle 𝒪(2,2)\mathcal{O}(-2,-2) of 1×1\mathbb{P}^{1}\times\mathbb{P}^{1} in the presence of topological D-branes was computed using the refined topological vertex formalism. Then these partition functions are proved to be equal to the quiver partition function (17) for r=1r=1 and r=2r=2 respectively. More precisely one has to expand the open string partition function in terms of holonomy variables, say xix_{i}, that parametrize D-brane boundary conditions, and take the coefficient of xidx_{i}^{d}. Then this coefficient is to be compared with the gauge theory or quiver partition function.i.e. for r=1r=1

Zquiverd,4d(q1,q2,T)=Zopen,dref(q1,q2,T)\displaystyle Z^{d,4d}_{quiver}(q_{1},q_{2},T)=Z^{ref}_{open,d}(q_{1},q_{2},T) (18)

and for r=2r=2

Zquiverd,4d(q1,q2,ρ1,ρ2,T)\displaystyle Z^{d,4d}_{quiver}(q_{1},q_{2},\rho_{1},\rho_{2},T) =\displaystyle= Zopen,dref(q1,q2,ρ12,T)\displaystyle Z^{ref}_{open,d}(q_{1},q_{2},\rho_{12},T)

where the topological string partition function Zopen,drefZ^{ref}_{open,d} is normalised by dividing out by the gauge theory perturbative part. We elaborate on the refined topological vertex computation in the next section (3) for the ’compactified’ geometries. It is important to note that in computing the open string partition function on the resolved conifold the defect brane was put on the un-preferred direction and the preferred direction was chosen to be the internal one. Moreover in our case the lagrangian brane is put along an external leg. The open topological partition function contains both perturbative and non-perturbative parts of gauge theory. In making the comparison (18) one has to exclude the perturbative part.

3 Generating function of χy\chi_{y} genus

Instanton partition functions for gauge theories in the presence of surface defect in 4d, 5d and 6d are the generating functions of Euler characteristics, χy\chi_{y}-genera and elliptic genera of the moduli space under consideration Li:2004ef . For refined topological strings the open string defect amplitude can be written in four ways depending on whether the topological brane extends along ϵ12\mathbb{R}^{2}_{\epsilon_{1}} or ϵ22\mathbb{R}^{2}_{\epsilon_{2}} and whether it is put along the external non-compact leg or the internal compact leg Kozcaz:2018ndf .
In the M-theory lift of the type IIA topological strings, the equivariant Euler characteristics gets lifted to the χy\chi_{y} genus. It is a 5d defect gauge theory compactified on a circle S1S^{1}. The defect BPS states in M-theory framework are related to M2-brane BPS state counting. Consequently we write down the generating function for the χy\chi_{y} genus or 5d5d defect partition function as

Zdquiver(q1,q2,ρa,y,Q)\displaystyle Z^{quiver}_{d}(q_{1},q_{2},\rho_{a},y,Q) =\displaystyle= kQk|ν|=ka,b=1r\displaystyle\sum_{k}Q^{k}\sum_{|\nu|=k}\prod_{a,b=1}^{r} (20)
×\displaystyle\times (i,j)νa(1yρaρb1q1(νb)jtq2jνia1)(i,j)νb(1yρaρb1q1(νa)jtq2jνibj)(i,j)νa(1ρaρb1q1(νb)jtq2jνia1)(i,j)νb(1ρaρb1q1(νa)jtq2jνibj)\displaystyle\frac{\prod_{(i,j)\in\nu^{a}}(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{(\nu^{b})^{t}_{j}}q_{2}^{j-\nu_{i}^{a}-1})\prod_{(i,j)\in\nu^{b}}(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{(\nu^{a})^{t}_{j}}q_{2}^{j-\nu_{i}^{b}-j})}{\prod_{(i,j)\in\nu^{a}}(1-\rho_{a}\rho_{b}^{-1}q_{1}^{(\nu^{b})^{t}_{j}}q_{2}^{j-\nu_{i}^{a}-1})\prod_{(i,j)\in\nu^{b}}(1-\rho_{a}\rho_{b}^{-1}q_{1}^{(\nu^{a})^{t}_{j}}q_{2}^{j-\nu_{i}^{b}-j})}
×\displaystyle\times (μ¯,ν¯)|μ¯|=|ν¯|+d[(a=1ri=1cas=1μiaνiaρaq11iq2νias+1)\displaystyle\sum_{\begin{subarray}{c}(\underline{\mu},\underline{\nu})\\ |\underline{\mu}|=|\underline{\nu}|+d\end{subarray}}\Bigg{[}(\prod_{a=1}^{r}\prod_{i=1}^{c^{a}}\prod_{s=1}^{\mu^{a}_{i}-\nu^{a}_{i}}\rho_{a}q_{1}^{1-i}q_{2}^{-\nu^{a}_{i}-s+1})
×\displaystyle\times a,b=1ri=2ea+1j=1cbs=1μjbνjb(1yρaρb1q1jiq2νjb+sμia1)a,b=1ri=2ea+1j=1cbs=1μjbνjb(1ρaρb1q1jiq2νjb+sμia1)\displaystyle\frac{\prod_{a,b=1}^{r}\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu_{j}^{b}-\nu_{j}^{b}}(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu^{b}_{j}+s-\mu^{a}_{i}-1})}{\prod_{a,b=1}^{r}\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu_{j}^{b}-\nu_{j}^{b}}(1-\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu^{b}_{j}+s-\mu^{a}_{i}-1})}
×\displaystyle\times a,b=1ri=2ea+1j=1cbs=1μjbνjb(1ρaρb1q1jiq2νjb+sνi1a1)a,b=1ri=2ea+1j=1cbs=1μjbνjb(1yρaρb1q1jiq2νjb+sνi1a1)\displaystyle\frac{\prod_{a,b=1}^{r}\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu^{b}_{j}}(1-\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu^{b}_{j}+s-\nu^{a}_{i-1}-1})}{\prod_{a,b=1}^{r}\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu^{b}_{j}}(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu^{b}_{j}+s-\nu^{a}_{i-1}-1})}
×\displaystyle\times a,b=1rj=1cbs=1μjbνjb(1yρaρb1q1j1q2νjb+sμ1a1)a,b=1rj=1cbs=1μjbνjb(1ρaρb1q1j1q2νjb+sμ1a1)]\displaystyle\frac{\prod_{a,b=1}^{r}\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu^{b}_{j}}(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{j-1}q_{2}^{-\nu^{b}_{j}+s-\mu^{a}_{1}-1})}{\prod_{a,b=1}^{r}\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu^{b}_{j}}(1-\rho_{a}\rho_{b}^{-1}q_{1}^{j-1}q_{2}^{-\nu^{b}_{j}+s-\mu^{a}_{1}-1})}\Bigg{]}
\displaystyle\equiv kQk|ν|=k\displaystyle\sum_{k}Q^{k}\sum_{|\nu|=k}
×\displaystyle\times (i,j)νa(1yq1(νb)jtq2jνia1)(i,j)νb(1yq1(νa)jtq2jνibj)(i,j)νa(1q1(νb)jtq2jνia1)(i,j)νb(1q1(νa)jtq2jνibj)\displaystyle\frac{\prod_{(i,j)\in\nu^{a}}(1-yq_{1}^{(\nu^{b})^{t}_{j}}q_{2}^{j-\nu_{i}^{a}-1})\prod_{(i,j)\in\nu^{b}}(1-yq_{1}^{(\nu^{a})^{t}_{j}}q_{2}^{j-\nu_{i}^{b}-j})}{\prod_{(i,j)\in\nu^{a}}(1-q_{1}^{(\nu^{b})^{t}_{j}}q_{2}^{j-\nu_{i}^{a}-1})\prod_{(i,j)\in\nu^{b}}(1-q_{1}^{(\nu^{a})^{t}_{j}}q_{2}^{j-\nu_{i}^{b}-j})}
×\displaystyle\times (μ¯,ν¯)|μ¯|=|ν¯|+dW(μ¯,ν¯)(q1,q2,ρa,y)\displaystyle\sum_{\begin{subarray}{c}(\underline{\mu},\underline{\nu})\\ |\underline{\mu}|=|\underline{\nu}|+d\end{subarray}}W_{(\underline{\mu},\underline{\nu})}(q_{1},q_{2},\rho_{a},y)

which for r=1r=1 becomes

Zd5d,quiver(q1,q2,ρ1,Q)\displaystyle Z^{5d,quiver}_{d}(q_{1},q_{2},\rho_{1},Q) =\displaystyle= kQkχy(M,k,q1,q2,ρ1))\displaystyle\sum_{k}Q^{k}\chi_{y}(M,k,q_{1},q_{2},\rho_{1})) (21)
=\displaystyle= kQk|ν|=k\displaystyle\sum_{k}Q^{k}\sum_{|\nu|=k}
×\displaystyle\times (i,j)ν(1yq1(ν)jtq2jνi1)(i,j)ν(1yq1(ν)jtq2jνij)(i,j)ν(1q1(ν)jtq2jνi1)(i,j)ν(1q1(ν)jtq2jνij)\displaystyle\frac{\prod_{(i,j)\in\nu}(1-yq_{1}^{(\nu)^{t}_{j}}q_{2}^{j-\nu_{i}-1})\prod_{(i,j)\in\nu}(1-yq_{1}^{(\nu)^{t}_{j}}q_{2}^{j-\nu_{i}-j})}{\prod_{(i,j)\in\nu}(1-q_{1}^{(\nu)^{t}_{j}}q_{2}^{j-\nu_{i}-1})\prod_{(i,j)\in\nu}(1-q_{1}^{(\nu)^{t}_{j}}q_{2}^{j-\nu_{i}-j})}
×\displaystyle\times (μ,ν)|μ|=|ν|+d[i=1cs=1μiνiρ1q11iq2νis+1\displaystyle\sum_{\begin{subarray}{c}(\mu,\nu)\\ |\mu|=|\nu|+d\end{subarray}}\Bigg{[}\prod_{i=1}^{c}\prod_{s=1}^{\mu_{i}-\nu_{i}}\rho_{1}q_{1}^{1-i}q_{2}^{-\nu_{i}-s+1}
×\displaystyle\times i=2e+1j=1cs=1μjνj(1yq1jiq2νj+sμi1)i=2e+1j=1cs=1μjνj(1q1jiq2νj+sμi1)\displaystyle\frac{\prod_{i=2}^{e+1}\prod_{j=1}^{c}\prod_{s=1}^{\mu_{j}-\nu_{j}}(1-yq_{1}^{j-i}q_{2}^{\nu_{j}+s-\mu_{i}-1})}{\prod_{i=2}^{e+1}\prod_{j=1}^{c}\prod_{s=1}^{\mu_{j}-\nu_{j}}(1-q_{1}^{j-i}q_{2}^{\nu_{j}+s-\mu_{i}-1})}
×\displaystyle\times i=2e+1j=1cs=1μjνj(1q1jiq2νj+sνi11)i=2e+1j=1cs=1μjνj(1yq1jiq2νj+sνi11)\displaystyle\frac{\prod_{i=2}^{e+1}\prod_{j=1}^{c}\prod_{s=1}^{\mu_{j}-\nu_{j}}(1-q_{1}^{j-i}q_{2}^{\nu_{j}+s-\nu_{i-1}-1})}{\prod_{i=2}^{e+1}\prod_{j=1}^{c}\prod_{s=1}^{\mu_{j}-\nu_{j}}(1-yq_{1}^{j-i}q_{2}^{\nu_{j}+s-\nu_{i-1}-1})}
×\displaystyle\times j=1cs=1μjνj(1yq1j1q2νj+sμ11)j=1cs=1μjνj(1q1j1q2νj+sμ11)]\displaystyle\frac{\prod_{j=1}^{c}\prod_{s=1}^{\mu_{j}-\nu_{j}}(1-yq_{1}^{j-1}q_{2}^{-\nu_{j}+s-\mu_{1}-1})}{\prod_{j=1}^{c}\prod_{s=1}^{\mu_{j}-\nu_{j}}(1-q_{1}^{j-1}q_{2}^{-\nu_{j}+s-\mu_{1}-1})}\Bigg{]}
\displaystyle\equiv kQk|ν|=k\displaystyle\sum_{k}Q^{k}\sum_{|\nu|=k}
×\displaystyle\times (i,j)νa(1yq1(ν)jtq2jνi1)(i,j)ν(1yq1(ν)jtq2jνij)(i,j)νa(1q1(ν)jtq2jνi1)(i,j)ν(1q1(ν)jtq2jνij)\displaystyle\frac{\prod_{(i,j)\in\nu^{a}}(1-yq_{1}^{(\nu)^{t}_{j}}q_{2}^{j-\nu_{i}-1})\prod_{(i,j)\in\nu}(1-yq_{1}^{(\nu)^{t}_{j}}q_{2}^{j-\nu_{i}-j})}{\prod_{(i,j)\in\nu^{a}}(1-q_{1}^{(\nu)^{t}_{j}}q_{2}^{j-\nu_{i}-1})\prod_{(i,j)\in\nu}(1-q_{1}^{(\nu)^{t}_{j}}q_{2}^{j-\nu_{i}-j})}
×\displaystyle\times (μ,ν)|μ|=|ν|+dW(μ,ν)(q1,q2,ρ1,y)\displaystyle\sum_{\begin{subarray}{c}(\mu,\nu)\\ |\mu|=|\nu|+d\end{subarray}}W_{(\mu,\nu)}(q_{1},q_{2},\rho_{1},y)

Here Wν,d(q1,q2,ρ1,y)(μ,ν)|μ|=|ν|+dW(μ,ν)(q1,q2,ρ1,y)W_{\nu,d}(q_{1},q_{2},\rho_{1},y)\equiv\sum_{\begin{subarray}{c}(\mu,\nu)\\ |\mu|=|\nu|+d\end{subarray}}W_{(\mu,\nu)}(q_{1},q_{2},\rho_{1},y) contains the information about the surface defects and the BPS states bound to it.

3.1 Open string/defect brane partition function 𝒪(1)𝒪(1)1\mathcal{O}(-1)\oplus\mathcal{O}(-1)\to\mathbb{P}^{1}: gauging the two external legs

Q2Q_{2}Q1Q_{1}
Figure 4: partially compactified toric diagram of resolved conifold with a Lagrangian brane

The resolved conifold i.e. the total space of the bundle 𝒪(1)𝒪(1)1\mathcal{O}(-1)\oplus\mathcal{O}(-1)\to\mathbb{P}^{1} can be parametrised by the pair of coordinates (y1,y2)(y_{1},y_{2}) and (y3,y4)(y_{3},y_{4}). These pairs of coordinates define two line bundles on 1\mathbb{P}^{1} as follows

y1=ζy2,y3=ζ1y4\displaystyle y_{1}=\zeta y_{2},\quad y_{3}=\zeta^{-1}y_{4} (22)

for ζ1\zeta\in\mathbb{P}^{1}. The partial compactification of the corresponding web diagram imposes periodic boundary conditions on the external legs.
Note that in the toric diagram (4) two blue lines indicate the identification of vertical edges, the green line intersects the preferred direction and the red dotted line denotes the lagrangian brane. For our purposes it is more useful to choose the internal line as our preferred direction. With the following definitions of the framing factor fμ(t,q)f_{\mu}(t,q) and the refined topological vertex Cλμν(t,q)C_{\lambda\mu\nu}(t,q) given by Bruzzo:2010fk

fμ(t,q)\displaystyle f_{\mu}(t,q) =\displaystyle= (1)|μ|tμt/2|μ|/2qμ2/2+|μ|/2\displaystyle(-1)^{|\mu|}t^{||\mu^{t}||/2-|\mu|/2}q^{-||\mu||^{2}/2+|\mu|/2}
Cλμν(t,q)\displaystyle C_{\lambda\mu\nu}(t,q) =\displaystyle= (tq)μ22qκ(μ)+ν22Z~ν(t,q)\displaystyle(\frac{t}{q})^{\frac{||\mu||^{2}}{2}}q^{\frac{\kappa(\mu)+||\nu||^{2}}{2}}\tilde{Z}_{\nu}(t,q)
×\displaystyle\times η(qt)|η|+|λ||μ|2sλt/η(tρqν)sμ/η(tνtqρ)\displaystyle\sum_{\eta}(\frac{q}{t})^{\frac{|\eta|+|\lambda|-|\mu|}{2}}s_{\lambda^{t}/\eta}(t^{-\rho}q^{-\nu})s_{\mu/\eta}(t^{-\nu^{t}}q^{-\rho})
Z~ν(t,q)\displaystyle\tilde{Z}_{\nu}(t,q) =\displaystyle= i=1l(ν)j=1λi(1qλijtλjti+1)1\displaystyle\prod_{i=1}^{l(\nu)}\prod_{j=1}^{\lambda_{i}}(1-q^{\lambda_{i}-j}t^{\lambda_{j}^{t}-i+1})^{-1}
ρ\displaystyle\rho =\displaystyle= {12,32,52,},tρqν={t12qλ1,t32qλ2,t52qλ3,}\displaystyle\{-\frac{1}{2},-\frac{3}{2},-\frac{5}{2},...\},\quad t^{-\rho}q^{-\nu}=\{t^{\frac{1}{2}}q^{-\lambda_{1}},t^{\frac{3}{2}}q^{-\lambda_{2}},t^{\frac{5}{2}}q^{-\lambda_{3}},...\} (23)

the refined amplitude can be written in the following form

Zopenref\displaystyle Z^{ref}_{open} =\displaystyle= sλ(x)𝒵λ(Q1,Q2,t,q)\displaystyle\sum s_{\lambda}(x)\mathcal{Z}_{\lambda}(Q_{1},Q_{2},t,q)
=\displaystyle= νsλ(x)μ,λ(Q1)|ν|(Q2)|μ|Cμν(t,q)Cλμtνt(q,t)\displaystyle\sum_{\nu}s_{\lambda}(x)\sum_{\mu,\lambda}(-Q_{1})^{|\nu|}(-Q_{2})^{|\mu|}C_{\emptyset\mu\nu}(t,q)C_{\lambda\mu^{t}\nu^{t}}(q,t)
=\displaystyle= ν(Q1)|ν|(qt)ν2νt22Pνt(tρ;q,t)Pν(qρ;t,q)i,j(1+(txj)tνitqi1)\displaystyle\sum_{\nu}(-Q_{1})^{|\nu|}(\frac{q}{t})^{\frac{||\nu||^{2}-||\nu^{t}||^{2}}{2}}P_{\nu^{t}}(t^{-\rho};q,t)P_{\nu}(q^{-\rho};t,q)\prod_{i,j}(1+(\sqrt{t}x_{j})t^{-\nu_{i}^{t}}q^{i-1})
×\displaystyle\times k,l(1Q2tνktρlqρkνl)i,j(1+Q2qt(txj)tνitqi1)1\displaystyle\prod_{k,l}(1-Q_{2}t^{-\nu^{t}_{k}-\rho_{l}}q^{-\rho_{k}-\nu_{l}})\prod_{i,j}(1+Q_{2}\sqrt{\frac{q}{t}}(\sqrt{t}x_{j})t^{-\nu_{i}^{t}}q^{i-1})^{-1}

where

Pνt(tρ;q,t)\displaystyle P_{\nu^{t}}(t^{-\rho};q,t) =\displaystyle= tν22Z~ν(t,q)\displaystyle t^{\frac{||\nu||^{2}}{2}}\tilde{Z}_{\nu}(t,q) (25)
=\displaystyle= tν22(i,j)ν(1ta(i,j)+1ql(i,j))1\displaystyle t^{\frac{||\nu||^{2}}{2}}\prod_{(i,j)\in\nu}(1-t^{a(i,j)+1}q^{l(i,j)})^{-1}
=\displaystyle= tν22(i,j)ν(1tvjti+1qνij)1\displaystyle t^{\frac{||\nu||^{2}}{2}}\prod_{(i,j)\in\nu}(1-t^{v_{j}^{t}-i+1}q^{\nu_{i}-j})^{-1}

After dividing by the gauge theory perturbative factor k,l(1Q2tρlqρk)\prod_{k,l}(1-Q_{2}t^{-\rho_{l}}q^{-\rho_{k}}) and using the following identity Iqbal:2007ii

k,l(1Q2tμktρlqρkμl)k,l(1Q2tρlqρk)=sμ(1Q2ta(s)k12ql(s)k12)(1Q2ta(s)k+12ql(s)k+12)\displaystyle\frac{\prod_{k,l}(1-Q_{2}t^{-\mu^{t}_{k}-\rho_{l}}q^{-\rho_{k}-\mu_{l}})}{\prod_{k,l}(1-Q_{2}t^{-\rho_{l}}q^{-\rho_{k}})}=\prod_{s\in\mu}(1-Q_{2}t^{-a(s)_{k}-\frac{1}{2}}q^{-l(s)_{k}-\frac{1}{2}})(1-Q_{2}t^{a(s)_{k}+\frac{1}{2}}q^{l(s)_{k}+\frac{1}{2}}) (26)

where a(i,j)=νjtia(i,j)=\nu_{j}^{t}-i, l(i,j)=νijl(i,j)=\nu_{i}-j denote the arm length and leg length of a box at the position (i,j)(i,j) in the Young diagram, we get in the redefined variables yj=xjty_{j}=x_{j}\sqrt{t}

Z~openref\displaystyle\tilde{Z}^{ref}_{open} =\displaystyle= ν(Q1)|ν|(qt)ν2νt22Pνt(tρ;q,t)Pν(qρ;t,q)\displaystyle\sum_{\nu}(-Q_{1})^{|\nu|}(\frac{q}{t})^{\frac{||\nu||^{2}-||\nu^{t}||^{2}}{2}}P_{\nu^{t}}(t^{-\rho};q,t)P_{\nu}(q^{-\rho};t,q)
×\displaystyle\times sν(1Q2ta(s)k12ql(s)k12)(1Q2ta(s)k+12ql(s)k+12)\displaystyle\prod_{s\in\nu}(1-Q_{2}t^{-a(s)_{k}-\frac{1}{2}}q^{-l(s)_{k}-\frac{1}{2}})(1-Q_{2}t^{a(s)_{k}+\frac{1}{2}}q^{l(s)_{k}+\frac{1}{2}})
×\displaystyle\times i,j(1+yjtνitqi1)m,n(1+Q2qtyjtνitqi1)1\displaystyle\prod_{i,j}(1+y_{j}t^{-\nu_{i}^{t}}q^{i-1})\prod_{m,n}(1+Q_{2}\sqrt{\frac{q}{t}}y_{j}t^{-\nu_{i}^{t}}q^{i-1})^{-1}

Now to extract the contribution of the surface operator we proceed as suggested in Bruzzo:2010fk . The right hand side of the last equation can be expanded in the basis of symmetric functions, in particular in the basis of monomial symmetric functions mν(xi)m_{\nu}(x_{i}) MacDonald:2019755 . By definition, mn,0,0,(x)=x1n+x2n+m_{n,0,0,...}(\mbox{\bf{x}})=x_{1}^{n}+x_{2}^{n}+... for any positive integer nn. Then we will denote by Z~open,dref\tilde{Z}^{ref}_{open,d} the coefficient of md,0,0,(𝐱)m_{d,0,0,...}(\bf{x}) in the expansion.
Next note that

ln(i,j(1+yjtνitqi1))=i=1l=1(1)l1l(k=1tkνitqk(i1)ek(y))l\displaystyle\mbox{ln}\big{(}\prod_{i,j}(1+y_{j}t^{-\nu_{i}^{t}}q^{i-1})\big{)}=\sum_{i=1}^{\infty}\sum_{l=1}^{\infty}\frac{(-1)^{l-1}}{l}(\sum_{k=1}^{\infty}t^{-k\nu_{i}^{t}}q^{k(i-1)}e_{k}(\mbox{\bf{y}}))^{l} (28)

where ek(𝐲)e_{k}(\bf{y}), k0k\in\mathbb{Z}_{\geq 0} denotes the degree kk elementary symmetric function in the variables (y1,y2,,)(y_{1},y_{2},...,). Recall the fact that for a partition λ\lambda and its conjugate partition λ\lambda^{\prime} we have the expansion MacDonald:2019755

eλ=mλ+μaλμmμ\displaystyle e_{\lambda^{\prime}}=m_{\lambda}+\sum_{\mu}a_{\lambda\mu}m_{\mu} (29)

where the summation over μ\mu is such that μ<λ\mu<\lambda and aλμa_{\lambda\mu} are non-negative integers. Using the identity (29) it is easy to see that the coefficient of md,0,0,(x)=x1d+x2d+m_{d,0,0,...}(\mbox{\bf{x}})=x_{1}^{d}+x_{2}^{d}+... in the expansion (28) can be obtained by restricting to k=1k=1

ei=1l=1(1)l1l(tνitq(i1)e1(y))l=el=1(1)l1li=1(tlνitql(i1)e1(y))l\displaystyle e^{\sum_{i=1}^{\infty}\sum_{l=1}^{\infty}\frac{(-1)^{l-1}}{l}(t^{-\nu_{i}^{t}}q^{(i-1)}e_{1}(\mbox{\bf{y}}))^{l}}=e^{\sum_{l=1}^{\infty}\frac{(-1)^{l-1}}{l}\sum_{i=1}^{\infty}(t^{-l\nu_{i}^{t}}q^{l(i-1)}e_{1}(\mbox{\bf{y}}))^{l}} (30)

Defining the quantity FνF_{\nu} by

Fν(q,t)=i=1qi1tνit\displaystyle F_{\nu}(q,t)=\sum_{i=1}^{\infty}q^{i-1}t^{-\nu_{i}^{t}} (31)

we can write

ei=1l=1(1)l1l(tνitq(i1)e1(y))l=el=1(1)l1lFν(ql,tl)e1(y))l\displaystyle e^{\sum_{i=1}^{\infty}\sum_{l=1}^{\infty}\frac{(-1)^{l-1}}{l}(t^{-\nu_{i}^{t}}q^{(i-1)}e_{1}(\mbox{\bf{y}}))^{l}}=e^{\sum_{l=1}^{\infty}\frac{(-1)^{l-1}}{l}F_{\nu}(q^{l},t^{l})e_{1}(\mbox{\bf{y}}))^{l}} (32)

From the last expression we find the coefficient of md,0,0,,,,m_{d,0,0,,,,} as

1d!η=(1d1,2d2,)d!k=1ddk!k=1d((1)k1kFν(q,t))\displaystyle\frac{1}{d!}\sum_{\eta=(1^{d_{1}},2^{d_{2}},...)}\frac{d!}{\prod_{k=1}^{d}d_{k}!}\prod_{k=1}^{d}\big{(}\frac{(-1)^{k-1}}{k}F_{\nu}(q,t)\big{)} (33)

where η=(1d1,2d2,)\eta=(1^{d_{1}},2^{d_{2}},...) denotes the set of all partitions of dd.
Following the same procedure we find Z~open,dref\tilde{Z}^{ref}_{open,d} as

Z~open,dref\displaystyle\tilde{Z}^{ref}_{open,d} =\displaystyle= μ(Q1)|μ|(qt)μ2μt22Pμt(tρ;q,t)Pμ(qρ;t,q)\displaystyle\sum_{\mu}(-Q_{1})^{|\mu|}(\frac{q}{t})^{\frac{||\mu||^{2}-||\mu^{t}||^{2}}{2}}P_{\mu^{t}}(t^{-\rho};q,t)P_{\mu}(q^{-\rho};t,q) (34)
×\displaystyle\times sμ(1Q2ta(s)k12ql(s)k12)(1Q2ta(s)k+12ql(s)k+12)\displaystyle\prod_{s\in\mu}(1-Q_{2}t^{-a(s)_{k}-\frac{1}{2}}q^{-l(s)_{k}-\frac{1}{2}})(1-Q_{2}t^{a(s)_{k}+\frac{1}{2}}q^{l(s)_{k}+\frac{1}{2}})
×\displaystyle\times η1=(1d1,2d2,,ddd)(1)dk=1ddkk=1d(dk!kdk)k=1dFμ(tk,qk,Q2)dk\displaystyle\sum_{\eta_{1}=(1^{d_{1}},2^{d_{2}},...,d^{d_{d}})}\frac{(-1)^{d-\sum_{k=1}^{d}d_{k}}}{\prod_{k=1}^{d}(d_{k}!k^{d_{k}})}\prod_{k=1}^{d}F_{\mu}(t^{k},q^{k},Q_{2})^{d_{k}}

where

Fν(t,q,Q1)\displaystyle F_{\nu}(t,q,Q_{1}) =\displaystyle= i=1qi1tνitQ2qti=1qi1tνit\displaystyle\sum_{i=1}q^{i-1}t^{-\nu_{i}^{t}}-Q_{2}\sqrt{\frac{q}{t}}\sum_{i=1}q^{i-1}t^{-\nu_{i}^{t}}

making a change of variables

q=q21,t=q1\displaystyle q=q_{2}^{-1},\quad t=q_{1} (36)
Fν(t,q,Q2)\displaystyle F_{\nu}(t,q,Q_{2}) =\displaystyle= ((i=1q21iq1νitQ2qti=1q21iq1νit))\displaystyle\bigg{(}(\sum_{i=1}q_{2}^{1-i}q_{1}^{-\nu^{t}_{i}}-Q_{2}\sqrt{\frac{q}{t}}\sum_{i=1}q_{2}^{1-i}q_{1}^{-\nu_{i}^{t}})\bigg{)}

The 5d generalisation of (18) states

Zd5d,quiver=Z~open,dref\displaystyle Z^{5d,quiver}_{d}=\tilde{Z}^{ref}_{open,d} (38)

or explicitly

kQk|ν|=k(i,j)ν(1yq1(ν)jtq2jνi1)(i,j)ν(1yq1(ν)jtq2jνij)(i,j)ν(1q1(ν)jtq2jνi1)(i,j)ν(1q1(ν)jtq2jνij)\displaystyle\sum_{k}Q^{k}\sum_{|\nu|=k}\frac{\prod_{(i,j)\in\nu}(1-yq_{1}^{(\nu)^{t}_{j}}q_{2}^{j-\nu_{i}-1})\prod_{(i,j)\in\nu}(1-yq_{1}^{(\nu)^{t}_{j}}q_{2}^{j-\nu_{i}-j})}{\prod_{(i,j)\in\nu}(1-q_{1}^{(\nu)^{t}_{j}}q_{2}^{j-\nu_{i}-1})\prod_{(i,j)\in\nu}(1-q_{1}^{(\nu)^{t}_{j}}q_{2}^{j-\nu_{i}-j})}
×(μ,ν)|μ|=|ν|+dW(μ,ν)(q1,q2,ρ1,y)=parameteridentification\displaystyle\times\sum_{\begin{subarray}{c}(\mu,\nu)\\ |\mu|=|\nu|+d\end{subarray}}W_{(\mu,\nu)}(q_{1},q_{2},\rho_{1},y)=_{parameter-identification}
×μ(Q1)|μ|(qt)μ2μt22Pμt(tρ;q,t)Pμ(qρ;t,q)\displaystyle\times\sum_{\mu}(-Q_{1})^{|\mu|}(\frac{q}{t})^{\frac{||\mu||^{2}-||\mu^{t}||^{2}}{2}}P_{\mu^{t}}(t^{-\rho};q,t)P_{\mu}(q^{-\rho};t,q)
×sμ(1Q2ta(s)k12ql(s)k12)(1Q2ta(s)k+12ql(s)k+12)\displaystyle\times\prod_{s\in\mu}(1-Q_{2}t^{-a(s)_{k}-\frac{1}{2}}q^{-l(s)_{k}-\frac{1}{2}})(1-Q_{2}t^{a(s)_{k}+\frac{1}{2}}q^{l(s)_{k}+\frac{1}{2}})
×η1(1)dk=1ddkk=1d(dk!kdk)k=1dFμ(tk,qk,Q2)dk\displaystyle\times\sum_{\eta_{1}}\frac{(-1)^{d-\sum_{k=1}^{d}d_{k}}}{\prod_{k=1}^{d}(d_{k}!k^{d_{k}})}\prod_{k=1}^{d}F_{\mu}(t^{k},q^{k},Q_{2})^{d_{k}} (39)

where parameteridentificationparameter-identification is described in the section 4. We have to divide by ZpertZ_{pert} since the refined topological vertex technique computes both perturbative and non-perturbative contributions from the gauge theory point of view, whereas Wν,d(q1,q2,y)W_{\nu,d}(q_{1},q_{2},y) only describes non-perturbative contributions.

3.2 special case of the conjecture

ν=\nu=\emptyset

We know that in the absence of a Lagrangian brane we have to set

ν=,\displaystyle\nu=\emptyset, (40)

and the conjecture (3.1) reduces to the special case

kQk|ν|=k(i,j)ν(1yq1(ν)jtq2jνi1)(i,j)ν(1yq1(ν)jtq2jνij)(i,j)ν(1q1(ν)jtq2jνi1)(i,j)ν(1q1(ν)jtq2jνij)=parameteridentification\displaystyle\sum_{k}Q^{k}\sum_{|\nu|=k}\frac{\prod_{(i,j)\in\nu}(1-yq_{1}^{(\nu)^{t}_{j}}q_{2}^{j-\nu_{i}-1})\prod_{(i,j)\in\nu}(1-yq_{1}^{(\nu)^{t}_{j}}q_{2}^{j-\nu_{i}-j})}{\prod_{(i,j)\in\nu}(1-q_{1}^{(\nu)^{t}_{j}}q_{2}^{j-\nu_{i}-1})\prod_{(i,j)\in\nu}(1-q_{1}^{(\nu)^{t}_{j}}q_{2}^{j-\nu_{i}-j})}=_{parameter-identification} (41)
×μ(Q1)|μ|(qt)μ2μt22Pμt(tρ;q,t)Pμ(qρ;t,q)\displaystyle\times\sum_{\mu}(-Q_{1})^{|\mu|}(\frac{q}{t})^{\frac{||\mu||^{2}-||\mu^{t}||^{2}}{2}}P_{\mu^{t}}(t^{-\rho};q,t)P_{\mu}(q^{-\rho};t,q)
×sμ(1Q2ta(s)k12ql(s)k12)(1Q2ta(s)k+12ql(s)k+12)\displaystyle\times\prod_{s\in\mu}(1-Q_{2}t^{-a(s)_{k}-\frac{1}{2}}q^{-l(s)_{k}-\frac{1}{2}})(1-Q_{2}t^{a(s)_{k}+\frac{1}{2}}q^{l(s)_{k}+\frac{1}{2}})

Using the following definition of the Macdonald symmetric function Pν(x;q,t)P_{\nu}(\mbox{\bf{x}};q,t)

Pνt(tρ;q,t)=tν22sν(1ta(s)+1ql(s))=tν22sν(1tl(s)+1qa(s))\displaystyle P_{\nu^{t}}(t^{-\rho};q,t)=t^{\frac{||\nu||^{2}}{2}}\prod_{s\in\nu}(1-t^{a(s)+1}q^{l(s)})=t^{\frac{||\nu||^{2}}{2}}\prod_{s\in\nu}(1-t^{l(s)+1}q^{a(s)}) (42)

the identity (41) is satisfied Iqbal:2007ii by taking

t=q1,q=q21,Q2=ytq,Q1=tqQ\displaystyle t=q_{1},\quad q=q_{2}^{-1},\quad Q_{2}=y\sqrt{\frac{t}{q}},\quad Q_{1}=\sqrt{\frac{t}{q}}Q (43)

ν\nu\neq\emptyset

From the 2d quiver quantum mechanical point of view yy is the mass deformation parameter Hollowood:2003gr and does not depend on ν\nu, the representation of the lagrangian branes. Thus turning on ν\nu from ν=\nu=\emptyset to ν\nu\neq\emptyset does not change the dependence on the mass parameter yy. Note that for y=1y=1 and q1=q21:=qq_{1}=q_{2}^{-1}:=q

Zd5d,quiver(q,ρa,Q)\displaystyle Z^{5d,quiver}_{d}(q,\rho_{a},Q) =\displaystyle= kQkχy(M,k,q,ρa))\displaystyle\sum_{k}Q^{k}\chi_{y}(M,k,q,\rho_{a})) (44)
\displaystyle\equiv kQk|ν|=k(μ¯,ν¯)|μ|=|ν|+d(1)\displaystyle\sum_{k}Q^{k}\sum_{|\nu|=k}\sum_{\begin{subarray}{c}(\underline{\mu},\underline{\nu})\\ |\mu|=|\nu|+d\end{subarray}}(1)

where the summation (μ¯,ν¯)|μ|=|ν|+d(1)\sum_{\begin{subarray}{c}(\underline{\mu},\underline{\nu})\\ |\mu|=|\nu|+d\end{subarray}}(1) counts the number of nested partitions (μa,νa)(\mu_{a},\nu_{a}) for a given ν¯\underline{\nu} such that
(a) no two points in the complements μa/νa\mu_{a}/\nu_{a} are in the same row
or in other words (μa,νa)(\mu_{a},\nu_{a}) and their respective number of columns (ca,ea)(c^{a},e^{a}) satisfy the following constraints
(b) 0caea1,0μiaνiaνi1aνia0\leq c^{a}-e^{a}\leq 1,\qquad 0\leq\mu_{i}^{a}-\nu_{i}^{a}\leq\nu_{i-1}^{a}-\nu_{i}^{a}  for 1ar1\leq a\leq r and i>0i>0,

3.3 Analytic Continuation

Although the contribution due to the topological brane does not seem to be polynomial in yy, it is easy to see that after analytically continuing q2q21q_{2}\to q_{2}^{-1}, Wν,d(q1,q2,y,ρa)W_{\nu,d}(q_{1},q_{2},y,\rho_{a}) can be written in the following form

W\displaystyle W (q1,q2,y,ρa)ν,d=(μ¯,ν¯)|μ¯|=|ν¯|+d[i=1cas=1μiaνiaρ1q11iq2νias+1a,b=1r{}_{\nu,d}(q_{1},q_{2},y,\rho_{a})=\sum_{\begin{subarray}{c}(\underline{\mu},\underline{\nu})\\ |\underline{\mu}|=|\underline{\nu}|+d\end{subarray}}\Bigg{[}\prod_{i=1}^{c^{a}}\prod_{s=1}^{\mu^{a}_{i}-\nu^{a}_{i}}\rho_{1}q_{1}^{1-i}q_{2}^{-\nu^{a}_{i}-s+1}\prod_{a,b=1}^{r} (45)
×\displaystyle\times (i=2ea+1j=1cbs=1μjbνjb(1yρaρb1q1jiq2νjb+sμia1)i=2ea+1j=1cbs=1μjbνjb(1ρaρb1q1jiq2νjb+sμia1)j=1cbs=1μjbνjb(1ρaρb1yq1j1q2νjb+sμ1a1)j=1cbs=1μjbνjb(1ρaρb1q1j1q2νjb+sμ1a1)\displaystyle\bigg{(}\frac{\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu^{b}_{j}}(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu_{j}^{b}+s-\mu^{a}_{i}-1})}{\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu^{b}_{j}}(1-\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu_{j}^{b}+s-\mu^{a}_{i}-1})}\frac{\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu^{b}_{j}}(1-\rho_{a}\rho_{b}^{-1}yq_{1}^{j-1}q_{2}^{-\nu^{b}_{j}+s-\mu^{a}_{1}-1})}{\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu^{b}_{j}}(1-\rho_{a}\rho_{b}^{-1}q_{1}^{j-1}q_{2}^{-\nu^{b}_{j}+s-\mu^{a}_{1}-1})}
×\displaystyle\times i=2ea+1j=1cbr=0(1yρaρb1q1jiq2νjbνi1ar1)(1yρaρb1q1jiq2μjbνi1a+r)i=2ea+1j=1cbr=0(1ρaρb1q1jiq2νjbνi1ar1)(1ρaρb1q1jiq2μjbνi1a+r))]\displaystyle\frac{\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{r=0}(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu_{j}^{b}-\nu_{i-1}^{a}-r-1})(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\mu_{j}^{b}-\nu_{i-1}^{a}+r})}{\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{r=0}(1-\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu_{j}^{b}-\nu_{i-1}^{a}-r-1})(1-\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\mu_{j}^{b}-\nu_{i-1}^{a}+r})}\bigg{)}\Bigg{]}
=\displaystyle= Wν,d(q1,q2,ρa)+𝒪(yd)\displaystyle W_{\nu,d}(q_{1},q_{2},\rho_{a})+\mathcal{O}(y^{d})

Noting that

i=1cas=1μiaνiaρ1q11iq2νias+1=q1q2d+(ν¯2μ¯2)\displaystyle\prod_{i=1}^{c^{a}}\prod_{s=1}^{\mu^{a}_{i}-\nu^{a}_{i}}\rho_{1}q_{1}^{1-i}q_{2}^{-\nu^{a}_{i}-s+1}=\sqrt{q_{1}q_{2}}^{d+(||\underline{\nu}||^{2}-||\underline{\mu}^{2}||)} (46)
W\displaystyle W (q1,q2,y,ρa)ν,d=(μ¯,ν¯)|μ¯|=|ν¯|+d[q1q2d+(ν¯2μ¯2)a,b=1r{}_{\nu,d}(q_{1},q_{2},y,\rho_{a})=\sum_{\begin{subarray}{c}(\underline{\mu},\underline{\nu})\\ |\underline{\mu}|=|\underline{\nu}|+d\end{subarray}}\Bigg{[}\sqrt{q_{1}q_{2}}^{d+(||\underline{\nu}||^{2}-||\underline{\mu}^{2}||)}\prod_{a,b=1}^{r} (47)
×\displaystyle\times (i=2ea+1j=1cbs=1μjbνjb(1yρaρb1q1jiq2νjb+sμia1)i=2ea+1j=1cbs=1μjbνjb(1ρaρb1q1jiq2νjb+sμia1)j=1cbs=1μjbνjb(1ρaρb1yq1j1q2νjb+sμ1a1)j=1cbs=1μjbνjb(1ρaρb1q1j1q2νjb+sμ1a1)\displaystyle\bigg{(}\frac{\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu^{b}_{j}}(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu_{j}^{b}+s-\mu^{a}_{i}-1})}{\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu^{b}_{j}}(1-\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu_{j}^{b}+s-\mu^{a}_{i}-1})}\frac{\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu^{b}_{j}}(1-\rho_{a}\rho_{b}^{-1}yq_{1}^{j-1}q_{2}^{-\nu^{b}_{j}+s-\mu^{a}_{1}-1})}{\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu^{b}_{j}}(1-\rho_{a}\rho_{b}^{-1}q_{1}^{j-1}q_{2}^{-\nu^{b}_{j}+s-\mu^{a}_{1}-1})}
×\displaystyle\times i=2ea+1j=1cbr=0(1yρaρb1q1jiq2νjbνi1ar1)(1yρaρb1q1jiq2μjbνi1a+r)i=2ea+1j=1cbr=0(1ρaρb1q1jiq2νjbνi1ar1)(1ρaρb1q1jiq2μjbνi1a+r))]\displaystyle\frac{\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{r=0}(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu_{j}^{b}-\nu_{i-1}^{a}-r-1})(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\mu_{j}^{b}-\nu_{i-1}^{a}+r})}{\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{r=0}(1-\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu_{j}^{b}-\nu_{i-1}^{a}-r-1})(1-\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\mu_{j}^{b}-\nu_{i-1}^{a}+r})}\bigg{)}\Bigg{]}

3.4 Generating function of χy\chi_{y} genus: r=2r=2

For this case the partially compactified toric diagram of the total space of the bundle 𝒪(2,2)\mathcal{O}(-2,-2) of 1×1\mathbb{P}^{1}\times\mathbb{P}^{1} is given in figure 5. Note that the edges which are identified are parallel.

ν\nuQmQ_{m}QfQ_{f}QbQ_{b}
Figure 5: partially compactified toric diagram of total space of the bundle 𝒪(2,2)\mathcal{O}(-2,-2) of 1×1\mathbb{P}^{1}\times\mathbb{P}^{1}
Q~1\tilde{Q}_{1}^{\prime}Q~2\tilde{Q}_{2}^{\prime}Q1Q_{1}^{\prime}QρQ_{\rho}^{\prime}Q2Q_{2}^{\prime}νt\nu^{t}
Figure 6: equivalent to toric diagram in figure 5

In our computation we choose the horizontal direction as the preferred one and hence the refined topological vertex can be used. An important ingredient in the refined vertex computation is the choice of preferred direction, which should be common to all of the vertices in the web-digram. However if we had chosen the direction along which the lagrangian brane is present as the preferred direction, it is not shared by two of the vertices in the web diagram (5). This requires the introduction of a new refined topological vertex Iqbal:2012mt .
To avoid the subtleties of the new refined topological vertex we can instead choose to compute refined partition function of the flopped geometry AIqbalsms ; Bastian:2018fba . Although for the horizontal preferred direction this is not necessary, we use it to illustrate the procedure. The flopped geometry is obtained from the original geometry by moving in the moduli space defined by the extended Kähler cone of the CY 3-fold under consideration. For instance the geometry defined in (22),has its flopped version defined as

y1=ζ~y4,y2=ζ~1y3\displaystyle y_{1}=\tilde{\zeta}y_{4},\quad y_{2}=\tilde{\zeta}^{-1}y_{3} (48)

for ζ~1\tilde{\zeta}\in\mathbb{P}^{1}. The toric Calabi Yau manifolds have the important property that they can be converted to a strip geometry form after appropriate number of blow ups and flop operations Iqbal:2004ne .
The flopped geometry contains as building blocks partially compactified 𝒪(1)𝒪(1)1\mathcal{O}(-1)\oplus\mathcal{O}(-1)\to\mathbb{P}^{1}s suitably glued together. To perform flop transition it is useful to draw the toric diagram in an equivalent way, figure 6, which makes the appearance of the building blocks 𝒪(1)𝒪(1)1\mathcal{O}(-1)\oplus\mathcal{O}(-1)\to\mathbb{P}^{1}, 𝒪(2)𝒪(0)1\mathcal{O}(-2)\oplus\mathcal{O}(0)\to\mathbb{P}^{1} and 𝒪(0)𝒪(2)1\mathcal{O}(0)\oplus\mathcal{O}(-2)\to\mathbb{P}^{1} manifest. Finally performing the flop on 1\mathbb{P}^{1} whose normal bundle is 𝒪(1)𝒪(1)\mathcal{O}(-1)\oplus\mathcal{O}(-1), results in the web diagram given in figure 7.

Q~1\tilde{Q}_{1}Q1Q_{1}QρQ_{\rho}Q2Q_{2}Q~2\tilde{Q}_{2}νt\nu^{t}
Figure 7: geometry after the flop operation in figure 6

The kähler parameters of the pre-flopped geometry to the flopped geometry are related as

Q1\displaystyle Q_{1} =\displaystyle= Q1Qρ,Q2=Q2Qρ,Q~1=Q~1Qρ,Q~2=Q~2Qρ\displaystyle Q^{\prime}_{1}Q_{\rho}^{\prime},\quad Q_{2}=Q^{\prime}_{2}Q_{\rho}^{\prime},\quad\tilde{Q}_{1}=\tilde{Q}^{\prime}_{1}Q_{\rho}^{\prime},\quad\tilde{Q}_{2}=\tilde{Q}^{\prime}_{2}Q_{\rho}^{\prime}
Qρ\displaystyle Q_{\rho} =\displaystyle= Qρ1,\displaystyle Q_{\rho}^{\prime-1},\quad (49)

The crucial result that allows the use of flop transition to compute the open topological string amplitude is the flop invariance of topological string computations by refined vertex technique Taki:2008hb ; Iqbal:2012xm ; Sugimoto:2015nha . If we denote the CY3-folds corresponding to web diagrams in figure 5 and figure 7 as X1,X2X_{1},X_{2}, the flop invariance implies

ZX2refined,open(Q1,Q2,Q~1,Q~2,Qρ,q,t;xi)\displaystyle Z^{refined,open}_{X_{2}}(Q_{1}^{\prime},Q_{2}^{\prime},\tilde{Q}^{\prime}_{1},\tilde{Q}^{\prime}_{2},Q^{\prime}_{\rho},q,t;x_{i})
=\displaystyle= ZX1refined,open(Q1Qρ,Q2Qρ,Q~1,Q~2,Qρ,Q~1Qρ,Q~2Qρ,Qρ1,q,t;xi)\displaystyle Z^{refined,open}_{X_{1}}(Q^{\prime}_{1}Q_{\rho}^{\prime},Q^{\prime}_{2}Q_{\rho}^{\prime},\tilde{Q}_{1},\tilde{Q}_{2},Q_{\rho},\tilde{Q}^{\prime}_{1}Q_{\rho}^{\prime},\tilde{Q}^{\prime}_{2}Q_{\rho}^{\prime},Q_{\rho}^{\prime-1},q,t;x_{i})

We get the following refined amplitude for the flopped geometry in figure 7 using the refined topological vertex formalism

Zopen,X2ref(Q1,Q2,Q~1,Q~2,Qρ,q,t;x)\displaystyle Z_{open,X_{2}}^{ref}(Q_{1},Q_{2},\tilde{Q}_{1},\tilde{Q}_{2},Q_{\rho},q,t;x) =\displaystyle= allindices(Q1)|μ1|(Q2)|μ2|(Q~1)|μ~1|(Q~2)|μ~2|(Qρ)|ρ|\displaystyle\sum_{all\quad indices}(-Q_{1})^{|\mu_{1}|}(-Q_{2})^{|\mu_{2}|}(-\tilde{Q}_{1})^{|\tilde{\mu}_{1}|}(-\tilde{Q}_{2})^{|\tilde{\mu}_{2}|}(-Q_{\rho})^{|\rho|} (51)
×\displaystyle\times Cνtμ2μ~2(t,q)Cρμ2tμ~2t(q,t)Cρtμ1tμ~1(t,q)Cμ1μ~1t(q,t)sνt(x)\displaystyle C_{\nu^{t}\mu_{2}\tilde{\mu}_{2}}(t,q)C_{\rho\mu_{2}^{t}\tilde{\mu}^{t}_{2}}(q,t)C_{\rho^{t}\mu_{1}^{t}\tilde{\mu}_{1}}(t,q)C_{\emptyset\mu_{1}\tilde{\mu}^{t}_{1}}(q,t)s_{\nu^{t}}(x)

Using the expression for the refined topological vertex (3.1) and using the following skew Schur function identities repeatedly

λsλ/α(x)sλ/β(y)\displaystyle\sum_{\lambda}s_{\lambda/\alpha}(x)s_{\lambda/\beta}(y) =\displaystyle= i,j(1xiyj)1sβ/η(x)sα/η(y)\displaystyle\prod_{i,j}(1-x_{i}y_{j})^{-1}\sum s_{\beta/\eta}(x)s_{\alpha/\eta}(y)
λsλt/α(x)sλ/β(y)\displaystyle\sum_{\lambda}s_{\lambda^{t}/\alpha}(x)s_{\lambda/\beta}(y) =\displaystyle= i,j(1+xiyj)sβt/ηt(x)sαt/η(y)\displaystyle\prod_{i,j}(1+x_{i}y_{j})\sum s_{\beta^{t}/\eta^{t}}(x)s_{\alpha^{t}/\eta}(y) (52)

we get the expression

Z\displaystyle Z (Q1,Q2,Q~1,Q~2,Qρ,q,t;x)open,X2ref=μ~1,μ~2(Q~1)|μ~1|(Q~2)|μ~2|Z~μ1~(t,q)Z~μ1t~(q,t)Z~μ2~(t,q)Z~μ2t~(q,t){}_{open,X_{2}}^{ref}(Q_{1},Q_{2},\tilde{Q}_{1},\tilde{Q}_{2},Q_{\rho},q,t;x)=\sum_{\tilde{\mu}_{1},\tilde{\mu}_{2}}(-\tilde{Q}_{1})^{|\tilde{\mu}_{1}|}(-\tilde{Q}_{2})^{|\tilde{\mu}_{2}|}\tilde{Z}_{\tilde{\mu_{1}}}(t,q)\tilde{Z}_{\tilde{\mu^{t}_{1}}}(q,t)\tilde{Z}_{\tilde{\mu_{2}}}(t,q)\tilde{Z}_{\tilde{\mu^{t}_{2}}}(q,t) (53)
×\displaystyle\times tμ~1t2+μ~2t2qμ~12+μ~22(1Q1qμ1~iρjtμ1t~jρi)(1Qρqμ1~jρitμ2t~iρj)\displaystyle\sqrt{t}^{||\tilde{\mu}^{t}_{1}||^{2}+||\tilde{\mu}^{t}_{2}||^{2}}\sqrt{q}^{||\tilde{\mu}_{1}||^{2}+||\tilde{\mu}_{2}||^{2}}\prod(1-Q_{1}q^{-\tilde{\mu_{1}}_{i}-\rho_{j}}t^{\tilde{-\mu^{t}_{1}}_{j}-\rho_{i}})\prod(1-Q_{\rho}q^{-\tilde{\mu_{1}}_{j}-\rho_{i}}t^{-\tilde{\mu^{t}_{2}}_{i}-\rho_{j}})
×\displaystyle\times (1Q2tμ2t~iρjqμ2~jρi)(1Q1Qρqttμ2t~jρiqμ1t~iρj)1\displaystyle\prod(1-Q_{2}t^{-\tilde{\mu_{2}^{t}}_{i}-\rho_{j}}q^{-\tilde{\mu_{2}}_{j}-\rho_{i}})\prod(1-Q_{1}Q_{\rho}\sqrt{\frac{q}{t}}t^{\tilde{-\mu_{2}^{t}}_{j}-\rho_{i}}q^{-\tilde{\mu^{t}_{1}}_{i}-\rho_{j}})^{-1}
×\displaystyle\times (1Q2Qρtqtμ2t~jρiqμ1~iρj)1(1Q1Q2Qρtμ2t~jρiqμ1t~iρj)\displaystyle\prod(1-Q_{2}Q_{\rho}\sqrt{\frac{t}{q}}t^{\tilde{-\mu_{2}^{t}}_{j}-\rho_{i}}q^{-\tilde{\mu_{1}}_{i}-\rho_{j}})^{-1}\prod(1-Q_{1}Q_{2}Q_{\rho}t^{\tilde{-\mu_{2}^{t}}_{j}-\rho_{i}}q^{-\tilde{\mu^{t}_{1}}_{i}-\rho_{j}})
×\displaystyle\times (1+xjqttρiqμ2~j)(1+Q2qtxitρjqμ2~j)1(1Q2Qρqtxitρjqμ1~j)1\displaystyle\prod(1+x_{j}\sqrt{\frac{q}{t}}t^{-\rho_{i}}q^{-\tilde{\mu_{2}}_{j}})\prod(1+Q_{2}\frac{q}{t}x_{i}t^{-\rho_{j}}q^{-\tilde{\mu_{2}}_{j}})^{-1}\prod(1-Q_{2}Q_{\rho}\sqrt{\frac{q}{t}}x_{i}t^{-\rho_{j}}q^{-\tilde{\mu_{1}}_{j}})^{-1}
×\displaystyle\times (1Q1Q2Qρqtxjtρiqμ1t~i)\displaystyle\prod(1-Q_{1}Q_{2}Q_{\rho}\frac{q}{t}x_{j}t^{-\rho_{i}}q^{-\tilde{\mu^{t}_{1}}_{i}})

Since the refined topological vertex formalism gives both perturbative and non-perturbative parts from gauge theory view point, we have to normalise 444Note that since for Q~1=Q~2=Qb\tilde{Q}_{1}=\tilde{Q}_{2}=Q_{b} the exponent of QbQ_{b} counts the instanton number, the gauge theory perturbative part is extracted by the limit Qb0Q_{b}\to 0. (53) to exclude the perturbative part. The normalised partition function turns out to be

Z^\displaystyle\hat{Z} (Q1,Q2,Q~1,Q~2,Qρ,q,t;x)open,X2ref=μ~1,μ~2(Q~1)|μ~1|(Q~2)|μ~2|Z~μ1~(t,q)Z~μ1t~(q,t)Z~μ2~(t,q)Z~μ2t~(q,t){}_{open,X_{2}}^{ref}(Q_{1},Q_{2},\tilde{Q}_{1},\tilde{Q}_{2},Q_{\rho},q,t;x)=\sum_{\tilde{\mu}_{1},\tilde{\mu}_{2}}(-\tilde{Q}_{1})^{|\tilde{\mu}_{1}|}(-\tilde{Q}_{2})^{|\tilde{\mu}_{2}|}\tilde{Z}_{\tilde{\mu_{1}}}(t,q)\tilde{Z}_{\tilde{\mu^{t}_{1}}}(q,t)\tilde{Z}_{\tilde{\mu_{2}}}(t,q)\tilde{Z}_{\tilde{\mu^{t}_{2}}}(q,t) (54)
×\displaystyle\times tμ~1t2+μ~2t2qμ~12+μ~22(1Q1qμ1~iρjtμ1t~jρi)(1Q1qρjtρi~)(1Qρqμ1~jρitμ2t~iρj)(1Qρqρitρj~)\displaystyle\sqrt{t}^{||\tilde{\mu}^{t}_{1}||^{2}+||\tilde{\mu}^{t}_{2}||^{2}}\sqrt{q}^{||\tilde{\mu}_{1}||^{2}+||\tilde{\mu}_{2}||^{2}}\frac{\prod(1-Q_{1}q^{-\tilde{\mu_{1}}_{i}-\rho_{j}}t^{\tilde{-\mu^{t}_{1}}_{j}-\rho_{i}})}{\prod(1-Q_{1}q^{-\rho_{j}}t^{\tilde{-\rho_{i}}})}\frac{\prod(1-Q_{\rho}q^{-\tilde{\mu_{1}}_{j}-\rho_{i}}t^{\tilde{-\mu^{t}_{2}}_{i}-\rho_{j}})}{\prod(1-Q_{\rho}q^{-\rho_{i}}t^{\tilde{-\rho_{j}})}}
×\displaystyle\times (1Q2tμ2t~iρjqμ2~jρi)(1Q2tρjqρi)(1Q1Qρqttρiqρj)(1Q1Qρqttμ2t~jρiqμ1t~iρj)\displaystyle\frac{\prod(1-Q_{2}t^{\tilde{-\mu_{2}^{t}}_{i}-\rho_{j}}q^{-\tilde{\mu_{2}}_{j}-\rho_{i}})}{\prod(1-Q_{2}t^{-\rho_{j}}q^{-\rho_{i}})}\frac{\prod(1-Q_{1}Q_{\rho}\sqrt{\frac{q}{t}}t^{-\rho_{i}}q^{-\rho_{j}})}{\prod(1-Q_{1}Q_{\rho}\sqrt{\frac{q}{t}}t^{\tilde{-\mu_{2}^{t}}_{j}-\rho_{i}}q^{-\tilde{\mu^{t}_{1}}_{i}-\rho_{j}})}
×\displaystyle\times (1Q2Qρtqtρiqρj)(1Q2Qρtqtμ2t~jρiqμ1~iρj)(1Q1Q2Qρtμ2t~jρiqμ1t~iρj)(1Q1Q2Qρtρiqρj)\displaystyle\frac{\prod(1-Q_{2}Q_{\rho}\sqrt{\frac{t}{q}}t^{-\rho_{i}}q^{-\rho_{j}})}{\prod(1-Q_{2}Q_{\rho}\sqrt{\frac{t}{q}}t^{\tilde{-\mu_{2}^{t}}_{j}-\rho_{i}}q^{-\tilde{\mu_{1}}_{i}-\rho_{j}})}\frac{\prod(1-Q_{1}Q_{2}Q_{\rho}t^{\tilde{-\mu_{2}^{t}}_{j}-\rho_{i}}q^{-\tilde{\mu^{t}_{1}}_{i}-\rho_{j}})}{\prod(1-Q_{1}Q_{2}Q_{\rho}t^{-\rho_{i}}q^{-\rho_{j}})}
×\displaystyle\times (1+xjqttρiqμ2~j)(1+Q2qtxitρjqμ2~j)1(1Q2Qρqtxitρjqμ1~j)1\displaystyle\prod(1+x_{j}\sqrt{\frac{q}{t}}t^{-\rho_{i}}q^{-\tilde{\mu_{2}}_{j}})\prod(1+Q_{2}\frac{q}{t}x_{i}t^{-\rho_{j}}q^{-\tilde{\mu_{2}}_{j}})^{-1}\prod(1-Q_{2}Q_{\rho}\sqrt{\frac{q}{t}}x_{i}t^{-\rho_{j}}q^{-\tilde{\mu_{1}}_{j}})^{-1}
×\displaystyle\times (1Q1Q2Qρqtxjtρiqμ1t~i)\displaystyle\prod(1-Q_{1}Q_{2}Q_{\rho}\frac{q}{t}x_{j}t^{-\rho_{i}}q^{-\tilde{\mu^{t}_{1}}_{i}})

A crucial step is to write the normalised partition function in terms of the Kähler parameters of the pre-flopped geometry (5) using (3.4)

Z^\displaystyle\hat{Z} (Q1,Q2,Q~1,Q~2,Qρ,q,t;x)open,X1ref=μ~1,μ~2(Q~1Q~ρ)|μ~1|(Q~2Q~ρ)|μ~2|Z~μ1~(t,q)Z~μ1t~(q,t)Z~μ2~(t,q)Z~μ2t~(q,t){}_{open,X_{1}}^{ref}(Q_{1}^{\prime},Q_{2}^{\prime},\tilde{Q}^{\prime}_{1},\tilde{Q}^{\prime}_{2},Q_{\rho}^{\prime},q,t;x)=\sum_{\tilde{\mu}_{1},\tilde{\mu}_{2}}(-\tilde{Q}^{\prime}_{1}\tilde{Q}_{\rho}^{\prime})^{|\tilde{\mu}_{1}|}(-\tilde{Q}^{\prime}_{2}\tilde{Q}_{\rho}^{\prime})^{|\tilde{\mu}_{2}|}\tilde{Z}_{\tilde{\mu_{1}}}(t,q)\tilde{Z}_{\tilde{\mu^{t}_{1}}}(q,t)\tilde{Z}_{\tilde{\mu_{2}}}(t,q)\tilde{Z}_{\tilde{\mu^{t}_{2}}}(q,t) (55)
×\displaystyle\times tμ~1t2+μ~2t2qμ~12+μ~22(1Q1Qρqμ1~iρjtμ1t~jρi)(1Q1Qρqρjtρi~)(1(Qρ)1qμ1~jρitμ2t~iρj)(1(Qρ)1qρitρj~)\displaystyle\sqrt{t}^{||\tilde{\mu}^{t}_{1}||^{2}+||\tilde{\mu}^{t}_{2}||^{2}}\sqrt{q}^{||\tilde{\mu}_{1}||^{2}+||\tilde{\mu}_{2}||^{2}}\frac{\prod(1-Q_{1}^{\prime}Q_{\rho}^{\prime}q^{-\tilde{\mu_{1}}_{i}-\rho_{j}}t^{\tilde{-\mu^{t}_{1}}_{j}-\rho_{i}})}{\prod(1-Q_{1}^{\prime}Q_{\rho}^{\prime}q^{-\rho_{j}}t^{\tilde{-\rho_{i}}})}\frac{\prod(1-(Q_{\rho}^{\prime})^{-1}q^{-\tilde{\mu_{1}}_{j}-\rho_{i}}t^{\tilde{-\mu^{t}_{2}}_{i}-\rho_{j}})}{\prod(1-(Q_{\rho}^{\prime})^{-1}q^{-\rho_{i}}t^{\tilde{-\rho_{j}})}}
×\displaystyle\times (1Q2Qρtμ2t~iρjqμ2~jρi)(1Q2Qρtρjqρi)(1Q1qttρiqρj)(1Q1qttμ2t~jρiqμ1t~iρj)\displaystyle\frac{\prod(1-Q_{2}^{\prime}Q_{\rho}^{\prime}t^{\tilde{-\mu_{2}^{t}}_{i}-\rho_{j}}q^{-\tilde{\mu_{2}}_{j}-\rho_{i}})}{\prod(1-Q_{2}^{\prime}Q_{\rho}^{\prime}t^{-\rho_{j}}q^{-\rho_{i}})}\frac{\prod(1-Q_{1}^{\prime}\sqrt{\frac{q}{t}}t^{-\rho_{i}}q^{-\rho_{j}})}{\prod(1-Q_{1}^{\prime}\sqrt{\frac{q}{t}}t^{\tilde{-\mu_{2}^{t}}_{j}-\rho_{i}}q^{-\tilde{\mu^{t}_{1}}_{i}-\rho_{j}})}
×\displaystyle\times (1Q2tqtρiqρj)(1Q2tqtμ2t~jρiqμ1~iρj)(1Q1Q2Qρtμ2t~jρiqμ1t~iρj)(1Q1Q2Qρtρiqρj)\displaystyle\frac{\prod(1-Q_{2}^{\prime}\sqrt{\frac{t}{q}}t^{-\rho_{i}}q^{-\rho_{j}})}{\prod(1-Q_{2}^{\prime}\sqrt{\frac{t}{q}}t^{\tilde{-\mu_{2}^{t}}_{j}-\rho_{i}}q^{-\tilde{\mu_{1}}_{i}-\rho_{j}})}\frac{\prod(1-Q_{1}^{\prime}Q_{2}^{\prime}Q_{\rho}^{\prime}t^{\tilde{-\mu_{2}^{t}}_{j}-\rho_{i}}q^{-\tilde{\mu^{t}_{1}}_{i}-\rho_{j}})}{\prod(1-Q_{1}^{\prime}Q_{2}^{\prime}Q_{\rho}^{\prime}t^{-\rho_{i}}q^{-\rho_{j}})}
×\displaystyle\times (1+xjqttρiqμ2~j)(1+Q2Qρqtxitρjqμ2~j)1\displaystyle\prod(1+x_{j}\sqrt{\frac{q}{t}}t^{-\rho_{i}}q^{-\tilde{\mu_{2}}_{j}})\prod(1+Q_{2}^{\prime}Q_{\rho}^{\prime}\frac{q}{t}x_{i}t^{-\rho_{j}}q^{-\tilde{\mu_{2}}_{j}})^{-1}
×\displaystyle\times (1Q2qtxitρjqμ1~j)1(1Q1Q2Qρqtxjtρiqμ1t~i)\displaystyle\prod(1-Q_{2}^{\prime}\sqrt{\frac{q}{t}}x_{i}t^{-\rho_{j}}q^{-\tilde{\mu_{1}}_{j}})^{-1}\prod(1-Q_{1}^{\prime}Q_{2}^{\prime}Q_{\rho}^{\prime}\frac{q}{t}x_{j}t^{-\rho_{i}}q^{-\tilde{\mu^{t}_{1}}_{i}})

It is important to note that the last expression has to be expanded in terms of QρQ_{\rho}^{\prime} instead of (Qρ)1(Q_{\rho}^{\prime})^{-1} to prove its equivalence to (54). Note that for particular values

Q1Qρ=tq,\displaystyle Q_{1}^{\prime}Q_{\rho}^{\prime}=\sqrt{\frac{t}{q}},
Q2Qρ=tq\displaystyle Q_{2}^{\prime}Q_{\rho}^{\prime}=\sqrt{\frac{t}{q}} (56)

the expression (55) reduces to

Zopen,X1ref(Q1,Q2,Q~1,Q~2,Qρ,q,t;x)\displaystyle Z_{open,X_{1}}^{ref}(Q_{1}^{\prime},Q_{2}^{\prime},\tilde{Q}^{\prime}_{1},\tilde{Q}^{\prime}_{2},Q_{\rho}^{\prime},q,t;x) |Q1Qρ=tq,Q2Qρ=tq=\displaystyle|_{Q_{1}^{\prime}Q_{\rho}^{\prime}=\sqrt{\frac{t}{q}},Q_{2}^{\prime}Q_{\rho}^{\prime}=\sqrt{\frac{t}{q}}}=
μ~1,μ~2(Q~1Qρ)|μ~1|(Q~2Qρ)|μ~2|tμ~1t2+μ~2t2qμ~12+μ~22\displaystyle\sum_{\tilde{\mu}_{1},\tilde{\mu}_{2}}(-\tilde{Q}^{\prime}_{1}Q_{\rho}^{\prime})^{|\tilde{\mu}_{1}|}(-\tilde{Q}^{\prime}_{2}Q_{\rho}^{\prime})^{|\tilde{\mu}_{2}|}\sqrt{t}^{||\tilde{\mu}^{t}_{1}||^{2}+||\tilde{\mu}^{t}_{2}||^{2}}\sqrt{q}^{||\tilde{\mu}_{1}||^{2}+||\tilde{\mu}_{2}||^{2}}

Moreover for the geometric engineering of pure SU(2)SU(2) with zero Chern-Simons coefficient for web diagram in figure (5), as in our case, we should impose the restrictions

Q~1\displaystyle\tilde{Q}_{1}^{\prime} =\displaystyle= Q~2=Qb\displaystyle\tilde{Q}_{2}^{\prime}=Q_{b}
Q1\displaystyle Q_{1}^{\prime} =\displaystyle= Q2=Qf\displaystyle Q_{2}^{\prime}=Q_{f} (58)

Restricting to the identification of parameters given in (3.4), corresponding to pure SU(2)SU(2) gauge theory, we get

Zopenref(Qb,Qf,Qm,q,t;x)\displaystyle Z_{open}^{ref}(Q_{b},Q_{f},Q_{m},q,t;x) =\displaystyle= μ~1,μ~2(QbQm)|μ~1|(QbQm)|μ~2|Z~μ1~(t,q)Z~μ1t~(q,t)Z~μ2~(t,q)Z~μ2t~(q,t)\displaystyle\sum_{\tilde{\mu}_{1},\tilde{\mu}_{2}}(-Q_{b}Q_{m})^{|\tilde{\mu}_{1}|}(-Q_{b}Q_{m})^{|\tilde{\mu}_{2}|}\tilde{Z}_{\tilde{\mu_{1}}}(t,q)\tilde{Z}_{\tilde{\mu^{t}_{1}}}(q,t)\tilde{Z}_{\tilde{\mu_{2}}}(t,q)\tilde{Z}_{\tilde{\mu^{t}_{2}}}(q,t) (59)
×\displaystyle\times tμ~1t2+μ~2t2qμ~12+μ~22\displaystyle\sqrt{t}^{||\tilde{\mu}^{t}_{1}||^{2}+||\tilde{\mu}^{t}_{2}||^{2}}\sqrt{q}^{||\tilde{\mu}_{1}||^{2}+||\tilde{\mu}_{2}||^{2}}
×\displaystyle\times (1QfQmqμ1~iρjtμ1t~jρi)(1QfQmqρjtρi~)(1(Qm)1qμ1~jρitμ2t~iρj)(1(Qm)1qρitρj~)\displaystyle\frac{\prod(1-Q_{f}Q_{m}q^{-\tilde{\mu_{1}}_{i}-\rho_{j}}t^{\tilde{-\mu^{t}_{1}}_{j}-\rho_{i}})}{\prod(1-Q_{f}Q_{m}q^{-\rho_{j}}t^{\tilde{-\rho_{i}}})}\frac{\prod(1-(Q_{m})^{-1}q^{-\tilde{\mu_{1}}_{j}-\rho_{i}}t^{\tilde{-\mu^{t}_{2}}_{i}-\rho_{j}})}{\prod(1-(Q_{m})^{-1}q^{-\rho_{i}}t^{\tilde{-\rho_{j}})}}
×\displaystyle\times (1QfQmtμ2t~iρjqμ2~jρi)(1QfQmtρjqρi)(1Qfqttρiqρj)(1Qfqttμ2t~jρiqμ1t~iρj)\displaystyle\frac{\prod(1-Q_{f}Q_{m}t^{\tilde{-\mu_{2}^{t}}_{i}-\rho_{j}}q^{-\tilde{\mu_{2}}_{j}-\rho_{i}})}{\prod(1-Q_{f}Q_{m}t^{-\rho_{j}}q^{-\rho_{i}})}\frac{\prod(1-Q_{f}\sqrt{\frac{q}{t}}t^{-\rho_{i}}q^{-\rho_{j}})}{\prod(1-Q_{f}\sqrt{\frac{q}{t}}t^{\tilde{-\mu_{2}^{t}}_{j}-\rho_{i}}q^{-\tilde{\mu^{t}_{1}}_{i}-\rho_{j}})}
×\displaystyle\times (1Qftqtρiqρj)(1Qftqtμ2t~jρiqμ1~iρj)(1QfQfQmtμ2t~jρiqμ1t~iρj)(1QfQfQmtρiqρj)\displaystyle\frac{\prod(1-Q_{f}\sqrt{\frac{t}{q}}t^{-\rho_{i}}q^{-\rho_{j}})}{\prod(1-Q_{f}\sqrt{\frac{t}{q}}t^{\tilde{-\mu_{2}^{t}}_{j}-\rho_{i}}q^{-\tilde{\mu_{1}}_{i}-\rho_{j}})}\frac{\prod(1-Q_{f}Q_{f}Q_{m}t^{\tilde{-\mu_{2}^{t}}_{j}-\rho_{i}}q^{-\tilde{\mu^{t}_{1}}_{i}-\rho_{j}})}{\prod(1-Q_{f}Q_{f}Q_{m}t^{-\rho_{i}}q^{-\rho_{j}})}
×\displaystyle\times (1+xjqttρiqμ2~j)(1+QfQmqtxitρjqμ2~j)1\displaystyle\prod(1+x_{j}\sqrt{\frac{q}{t}}t^{-\rho_{i}}q^{-\tilde{\mu_{2}}_{j}})\prod(1+Q_{f}Q_{m}\frac{q}{t}x_{i}t^{-\rho_{j}}q^{-\tilde{\mu_{2}}_{j}})^{-1}
×\displaystyle\times (1Qfqtxitρjqμ1~j)1(1QfQfQmqtxjtρiqμ1t~i)\displaystyle\prod(1-Q_{f}\sqrt{\frac{q}{t}}x_{i}t^{-\rho_{j}}q^{-\tilde{\mu_{1}}_{j}})^{-1}\prod(1-Q_{f}Q_{f}Q_{m}\frac{q}{t}x_{j}t^{-\rho_{i}}q^{-\tilde{\mu^{t}_{1}}_{i}})

Similar to (3.4), choosing QfQm=tqQ_{f}Q_{m}=\sqrt{\frac{t}{q}} results in the simplifed expression

Zopenref(Qb,Qf,Qm,q,t;x)|QfQm=tq\displaystyle Z_{open}^{ref}(Q_{b},Q_{f},Q_{m},q,t;x)|_{Q_{f}Q_{m}=\sqrt{\frac{t}{q}}} =\displaystyle= μ~1,μ~2(QbQm)|μ~1|+|μ~2|tμ~1t2+μ~2t2qμ~12+μ~22\displaystyle\sum_{\tilde{\mu}_{1},\tilde{\mu}_{2}}(-Q_{b}Q_{m})^{|\tilde{\mu}_{1}|+|\tilde{\mu}_{2}|}\sqrt{t}^{||\tilde{\mu}^{t}_{1}||^{2}+||\tilde{\mu}^{t}_{2}||^{2}}\sqrt{q}^{||\tilde{\mu}_{1}||^{2}+||\tilde{\mu}_{2}||^{2}}

The topological string expression (59) is to be compared with the quiver moduli partition function (following (21) and (45) ) given as follows

Z\displaystyle Z (q1,q2,ρa,y,Q)dquiver=kQk|ν|=k{}^{quiver}_{d}(q_{1},q_{2},\rho_{a},y,Q)=\sum_{k}Q^{k}\sum_{|\nu|=k}
×\displaystyle\times (i,j)νa(1yq1(νb)jtq2jνia1)(i,j)νb(1yq1(νa)jtq2jνibj)(i,j)νa(1q1(νb)jtq2jνia1)(i,j)νb(1q1(νa)jtq2jνibj)\displaystyle\frac{\prod_{(i,j)\in\nu^{a}}(1-yq_{1}^{(\nu^{b})^{t}_{j}}q_{2}^{j-\nu_{i}^{a}-1})\prod_{(i,j)\in\nu^{b}}(1-yq_{1}^{(\nu^{a})^{t}_{j}}q_{2}^{j-\nu_{i}^{b}-j})}{\prod_{(i,j)\in\nu^{a}}(1-q_{1}^{(\nu^{b})^{t}_{j}}q_{2}^{j-\nu_{i}^{a}-1})\prod_{(i,j)\in\nu^{b}}(1-q_{1}^{(\nu^{a})^{t}_{j}}q_{2}^{j-\nu_{i}^{b}-j})}
×\displaystyle\times (μ¯,ν¯)|μ¯|=|ν¯|+d[(a=12i=1cas=1μiaνiaρaq11iq2νias+1)a,b=12\displaystyle\sum_{\begin{subarray}{c}(\underline{\mu},\underline{\nu})\\ |\underline{\mu}|=|\underline{\nu}|+d\end{subarray}}\Bigg{[}(\prod_{a=1}^{2}\prod_{i=1}^{c^{a}}\prod_{s=1}^{\mu^{a}_{i}-\nu^{a}_{i}}\rho_{a}q_{1}^{1-i}q_{2}^{-\nu^{a}_{i}-s+1})\prod_{a,b=1}^{2}
×\displaystyle\times (i=2ea+1j=1cbs=1μjbνjb(1yρaρb1q1jiq2νjb+sμia1)i=2ea+1j=1cbs=1μjbνjb(1ρaρb1q1jiq2νjb+sμia1)j=1cbs=1μjbνjb(1ρaρb1yq1j1q2νjb+sμ1a1)j=1cbs=1μjbνjb(1ρaρb1q1j1q2νjb+sμ1a1)\displaystyle\bigg{(}\frac{\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu^{b}_{j}}(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu_{j}^{b}+s-\mu^{a}_{i}-1})}{\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu^{b}_{j}}(1-\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu_{j}^{b}+s-\mu^{a}_{i}-1})}\frac{\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu^{b}_{j}}(1-\rho_{a}\rho_{b}^{-1}yq_{1}^{j-1}q_{2}^{-\nu^{b}_{j}+s-\mu^{a}_{1}-1})}{\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu^{b}_{j}}(1-\rho_{a}\rho_{b}^{-1}q_{1}^{j-1}q_{2}^{-\nu^{b}_{j}+s-\mu^{a}_{1}-1})}
×\displaystyle\times i=2ea+1j=1cbr=0(1yρaρb1q1jiq2νjbνi1ar1)(1yρaρb1q1jiq2μjbνi1a+r)i=2ea+1j=1cbr=0(1ρaρb1q1jiq2νjbνi1ar1)(1ρaρb1q1jiq2μjbνi1a+r))]\displaystyle\frac{\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{r=0}(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu_{j}^{b}-\nu_{i-1}^{a}-r-1})(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\mu_{j}^{b}-\nu_{i-1}^{a}+r})}{\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{r=0}(1-\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu_{j}^{b}-\nu_{i-1}^{a}-r-1})(1-\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\mu_{j}^{b}-\nu_{i-1}^{a}+r})}\bigg{)}\Bigg{]}
Zdquiver(q1,q2,ρa,y,Q)|y=1\displaystyle Z^{quiver}_{d}(q_{1},q_{2},\rho_{a},y,Q)|_{y=1} =\displaystyle= kQk|ν¯|=k(μ¯,ν¯)|μ¯|=|ν¯|+d[(a=12i=1cas=1μiaνiaρaq11iq2νias+1)]\displaystyle\sum_{k}Q^{k}\sum_{|\underline{\nu}|=k}\sum_{\begin{subarray}{c}(\underline{\mu},\underline{\nu})\\ |\underline{\mu}|=|\underline{\nu}|+d\end{subarray}}\Bigg{[}(\prod_{a=1}^{2}\prod_{i=1}^{c^{a}}\prod_{s=1}^{\mu^{a}_{i}-\nu^{a}_{i}}\rho_{a}q_{1}^{1-i}q_{2}^{-\nu^{a}_{i}-s+1})\Bigg{]}

4 Relations between the parameters in the conjecture

In this section we describe consistency conditions that lead to the complete identification of parameters from the two sides of the conjectures (3.1,64). We will also formulate the conjecture in a more general form that contains information about the holonomy observables, denoted by 𝐱\bf{x}, parametrising the lagrangian branes. For the unrefined case in the absence of lagrangian branes on the external toric legs see Chuang:2013wpa ; Chuang:2012dv . Note the following facts:

  • In the decompactification limit log(Q1)-log(Q_{1})\to\infty the 4d version of (3.1) imposes the following identifications Bruzzo:2010fk

t=q1,q=q21,Q2=Tq1q2\displaystyle t=q_{1},\quad q=q_{2}^{-1},\quad Q_{2}=T\sqrt{q_{1}q_{2}} (63)
  • By considering the special case ν=,y=1\nu=\emptyset,y=1 in (3.1) one must choose Q1=tqyQ_{1}=\sqrt{\frac{t}{q}}y for the identity to hold. Since from 2d2d sigma model point of view yy is the mass parameter, this identification of parameters does not change for ν\nu\neq\emptyset.

  • As we consider cases r2r\geq 2 the characters ρa,\rho_{a}, are related to the Kähler parameters log(Qfi),log(Qb)-log(Q_{f_{i}}),-log(Q_{b}) of the fiber and base directions .

Given these constraints , the generalisation of (3.1) to the rank=2rank=2 case will then be given by

Zdquiver(q1,q2,ρa,y,Q)=parameteridentificationZ~open,dref(Qf,Qb,Qm,q,t)\displaystyle Z^{quiver}_{d}(q_{1},q_{2},\rho_{a},y,Q)=_{parameter-identification}\tilde{Z}^{ref}_{open,d}(Q_{f},Q_{b},Q_{m},q,t) (64)

Note that after parameter identification q=q21,t=q1,Qf=ρ11ρ2q=q_{2}^{-1},t=q_{1},Q_{f}=\rho_{1}^{-1}\rho_{2}, the decompactification limit log(Qm)-log(Q_{m})\to\infty leads to the result (5.3) proved in Bruzzo:2010fk

Zdquiver(q1,q2,ρ1,ρ2,Q)=Z~open,d(q1,q2,ρ11ρ2,Q)\displaystyle Z^{quiver}_{d}(q_{1},q_{2},\rho_{1},\rho_{2},Q)=\tilde{Z}_{open,d}(q_{1},q_{2},\rho_{1}^{-1}\rho_{2},Q) (65)

4.1 Unrefined topological strings

A subclass of the quiver moduli spaces discussed in section (2) and its corresponding partition function was discussed in detail by Diconescu et al. Chuang:2012dv ; Chuang:2013wpa . This subclass of moduli spaces does not contain the moduli corresponding to the D4-branes, the open strings between D4-D6 branes and D4-D2 branes. This moduli space modulo certain equivalence relations was shown to be isomorphic to the nested Hilbert scheme of points on 2\mathbb{C}^{2}. This scheme, denoted by 𝒩(γ)\mathcal{N}(\gamma) depends on an ordered sequence γ={mai}0ik\gamma=\{m_{a_{i}}\}_{0\leq i\leq k} of positive integers and parametrize a sequence of ideals sheaves 0kk100\subset\mathcal{I}_{k}\subset\mathcal{I}_{k-1\subset...\subset\mathcal{I}_{0}} of zero dimensional subschemes Zi2Z_{i}\subset\mathbb{C}^{2} and the corresponding topological data χ(𝒪Zi)=j=0iγj\chi(\mathcal{O}_{Z_{i}})=\sum_{j=0}^{i}\gamma_{j} for 0il0\leq i\leq l.
The character valued partition function is given by the equivariant χy\chi_{y} genus of a bundle 𝒱\mathcal{V} on 𝒩(γ)\mathcal{N}(\gamma). The bundle is identical to the bundle (g,p)\mathcal{L}_{(g,p)} described earlier and given as

𝒱(γ)η(𝒱g,p)η(Trgdet(𝕍)p)\displaystyle\mathcal{V}(\gamma)\equiv\eta^{*}(\mathcal{V}_{g,p})\simeq\eta^{*}\big{(}T^{*}\mathcal{H}^{r\oplus g}\otimes det(\mathbb{V})^{p}\big{)} (66)

The existence of the morphism η:𝒩(γ)r\eta:\mathcal{N}(\gamma)\to\mathcal{H}^{r} to the Hilbert scheme of rr points on 2\mathbb{C}^{2} makes it possible to apply the equivariant localization. The fixed points are given by the monomial ideals that are in one-to-one correspondence with the partitions of rr.
More interesting is the appearance of the modified Kostka-Macdonald coefficients. It can be explained by the existence of a map from the nested Hilbert scheme
ρ:𝒩(1,1,,1)~r\rho:\mathcal{N}(1,1,...,1)\to\tilde{\mathcal{H}}^{r}555the number of 11s in (1,1,,1)(1,1,...,1) is equal to rr. to isospectral Hilbert scheme discussed in haiman2000hilbert . The web of maps is shown below in the figure (8) as a commutative diagram. With this commutative diagram in mind, it was shown that there exists two very important pushforward maps

ρredγ𝒪𝒩(γ)\displaystyle\rho^{\gamma}_{red*}\mathcal{O}_{\mathcal{N}_{(\gamma)}} =\displaystyle= 𝒪redγ~\displaystyle\mathcal{O}_{\tilde{\mathcal{H}_{red}^{\gamma}}}
πredγ𝒪~red\displaystyle\pi_{red*}^{\gamma}\mathcal{O}_{\tilde{\mathcal{H}}_{red}} \displaystyle\simeq (πred𝒪red~)Sγ=𝒫Sγ\displaystyle(\pi_{red*}\mathcal{O}_{\tilde{\mathcal{H}_{red}}})^{S_{\gamma}}=\mathcal{P}^{S_{\gamma}} (67)

Next we summarise a sequence of arguments that fixes the notation of this section and which lead to the expansion of the topological string partition function in terms of the modified Macdonald polynomials.
a)since 𝒩(1,1,,1)\mathcal{N}(1,1,...,1) is reduced, this implies the existence of the morphism
ρred:𝒩(1,1,,1)~redr\rho_{red}:\mathcal{N}(1,1,...,1)\to\tilde{\mathcal{H}}^{r}_{red},
b)ρred𝒪𝒩(1,1,1)𝒪~redr\rho_{red*}\mathcal{O}_{\mathcal{N}(1,1...,1)}\equiv\mathcal{O}_{\tilde{\mathcal{H}}^{r}_{red}},
c)the pushforward map πred𝒪~red\pi_{red*}\mathcal{O}_{\tilde{\mathcal{H}}_{red}} is a vector bundle 𝒫\mathcal{P} on the Hilbert scheme and is isomorphic to the pushforward map η𝒪𝒩(1,1,,1)\eta_{*}\mathcal{O}_{\mathcal{N}(1,1,...,1)},
d)consider the stabiliser SγSrS_{\gamma}\subset S_{r} of the partition γ\gamma with SrS_{r} the symmetric group of order rr. This shows that ~r\tilde{\mathcal{H}}^{r} furnishes a representation of SγS_{\gamma} by the restriction map, Sγ×~r~rS_{\gamma}\times\tilde{\mathcal{H}}^{r}\to\tilde{\mathcal{H}}^{r},
e)~γ\tilde{\mathcal{H}}^{\gamma} denotes the quotient of ~r\tilde{\mathcal{H}}^{r} by SγS_{\gamma},
f)there exists a morphism ργ:𝒩(γ)~γ\rho^{\gamma}:\mathcal{N}(\gamma)\to\tilde{\mathcal{H}}^{\gamma} which is also true for the corresponding reduced schemes ρredγ:𝒩(γ)~redγ\rho^{\gamma}_{red}:\mathcal{N}(\gamma)\to\tilde{\mathcal{H}}^{\gamma}_{red}.
As a consequence of the equations (4.1) in the 𝐓\bf{T}-equivariant framework we have

χyT(𝒩(γ),ηγ𝒱g,p)\displaystyle\chi_{y}^{T}(\mathcal{N}(\gamma),\eta^{\gamma*}\mathcal{V}_{g,p}) =\displaystyle= χyT(r,(𝒫Sγ×r𝒱g,p))\displaystyle\chi_{y}^{T}(\mathcal{H}^{r},(\mathcal{P}^{S_{\gamma}}\times_{\mathcal{H}^{r}}\mathcal{V}_{g,p}))
=\displaystyle= μΩμg,p(q1,q2,y)chT(𝒫μγ)=μΩμg,p(q1,q2,y)λKλ,γ~K~λ,μ(q1,q2)\displaystyle\sum_{\mu}\Omega_{\mu}^{g,p}(q_{1},q_{2},y)ch_{T}(\mathcal{P}_{\mu}^{\gamma})=\sum_{\mu}\Omega_{\mu}^{g,p}(q_{1},q_{2},y)\sum_{\lambda}K_{\lambda,\tilde{\gamma}}\tilde{K}_{\lambda,\mu}(q_{1},q_{2})

where γ~\tilde{\gamma} denotes an unordered partition of rr determined by the sequence γ\gamma, Kλ,γ~K_{\lambda,\tilde{\gamma}} are the Kostka numbers and K~λ,μ(q1,q2)\tilde{K}_{\lambda,\mu}(q_{1},q_{2}) are the modified Kostka-Macdonald coefficients and in the second last equality equivariant localization was used. The character valued (K-theoretic) partition function is then given by the generating function of the χy\chi_{y} genus

ZKr(q1,q2,y;x)=m0+m1++mr1=rmi0χyT(𝒩(γ),ηγ𝒱g,p)i=0r1xima\displaystyle Z_{K}^{r}(q_{1},q_{2},y;\mbox{\bf{x}})=\sum_{\begin{subarray}{c}m_{0}+m_{1}+...+m_{r-1}=r\\ m_{i}\in\mathbb{Z}_{\geq 0}\end{subarray}}\chi_{y}^{T}(\mathcal{N}(\gamma),\eta^{\gamma*}\mathcal{V}_{g,p})\prod_{i=0}^{r-1}x_{i}^{m_{a}} (69)

where the factor i=0r1xima\prod_{i=0}^{r-1}x_{i}^{m_{a}} denotes the expansion of the partition function in terms of the symmetric functions of the holonomy observables.
Using the equation (4.1) one gets

ZKr(q1,q2,y;x)=μΩμg,p(q1,q2,y)H~μ(q2,q1;x)\displaystyle Z_{K}^{r}(q_{1},q_{2},y;\mbox{\bf{x}})=\sum_{\mu}\Omega_{\mu}^{g,p}(q_{1},q_{2},y)\tilde{H}_{\mu}(q_{2},q_{1};\mbox{\bf{x}}) (70)

For the moduli space of this section .i.e. r(2)\mathcal{H}^{r}(\mathbb{C}^{2}) the equivariant localization yields

Ωμg,p(q1,q2,y)=μ(q1l()q2a())g1p(1yq1l()q2a()+1)g(1yq1l()+1q2a())g(1q1l()q2a()+1)(1q1l()+1q2a())\displaystyle\Omega_{\mu}^{g,p}(q_{1},q_{2},y)=\prod_{\mu}(q_{1}^{l(\square)}q_{2}^{a(\square)})^{g-1-p}\frac{(1-yq_{1}^{-l(\square)}q_{2}^{a(\square)+1})^{g}(1-yq_{1}^{l(\square)+1}q_{2}^{-a(\square)})^{g}}{(1-q_{1}^{-l(\square)}q_{2}^{a(\square)+1})(1-q_{1}^{l(\square)+1}q_{2}^{-a(\square)})} (71)

with a()a(\square) and l()l(\square) defined as the arm length and the leg length of a box in the Young diagram corresponding to μ\mu. Therefore

ZKr(q1,q2,y;x)=μμ(q1l()q2a())g1p(1yq1l()q2a()+1)g(1yq1l()+1q2a())g(1q1l()q2a()+1)(1q1l()+1q2a())H~μ(q2,q1;x)\displaystyle Z_{K}^{r}(q_{1},q_{2},y;\mbox{\bf{x}})=\sum_{\mu}\prod_{\mu}(q_{1}^{l(\square)}q_{2}^{a(\square)})^{g-1-p}\frac{(1-yq_{1}^{-l(\square)}q_{2}^{a(\square)+1})^{g}(1-yq_{1}^{l(\square)+1}q_{2}^{-a(\square)})^{g}}{(1-q_{1}^{-l(\square)}q_{2}^{a(\square)+1})(1-q_{1}^{l(\square)+1}q_{2}^{-a(\square)})}\tilde{H}_{\mu}(q_{2},q_{1};\mbox{\bf{x}})

As shown in Chuang:2010ii ; Chuang:2012dv a particular change of variables 666Unfortunately this change of variables does not have a conceptual derivation. It was worked out by the requirement that the following conjecture should hold:
DT partition function of the CY3-fold X=Instanton partition function of 5d SUSY gauge theory geometrically engineered by X.
motivated by the geometric engineering conjecture between the 5d supersymmetric gauge theory with eight super charges and the Donaldson thomas theory of CY3-fold X, relates the K-theoretic partition function to the unrefined open string invariants as

ZXtop,open(q,x,y)\displaystyle Z^{top,open}_{X}(q,\mbox{\bf{x}},y) =?\displaystyle=^{?} 1+r1ZKr(qy1,q1y1,(1)g1pygx)\displaystyle 1+\sum_{r\geq 1}Z_{K}^{r}(qy^{-1},q^{-1}y^{-1},(-1)^{g-1-p}y^{-g}\mbox{\bf{x}})
=\displaystyle= 1+μμ(ql()a()yl()a())p(qy1)2l()+2a()(1)p|μ|\displaystyle 1+\sum_{\mu\neq\emptyset}\prod_{\square\in\mu}(q^{l(\square)-a(\square)}y^{-l(\square)-a(\square)})^{p}(qy^{-1})^{2l(\square)+2a(\square)}(-1)^{p|\mu|}
×\displaystyle\times (1yl()a()qa()l()1)2g(1yl()a()1qa()l()1)(1yl()a()+1qa()l()1)\displaystyle\frac{(1-y^{l(\square)-a(\square)}q^{-a(\square)-l(\square)-1})^{2g}}{(1-y^{l(\square)-a(\square)-1}q^{-a(\square)-l(\square)-1})(1-y^{l(\square)-a(\square)+1}q^{-a(\square)-l(\square)-1})}
×\displaystyle\times H~μ(q1y1,qy1,𝐱)\displaystyle\tilde{H}_{\mu}(q^{-1}y^{-1},qy^{-1},\bf{x})

The quantity ZXtop,open(q,x,y)Z^{top,open}_{X}(q,\mbox{\bf{x}},y) can be independently computed using the topological vertex formalism. It turns out Chuang:2012dv e.g. for x={Q,0,0.,,,}\mbox{\bf{x}}=\{Q,0,0.,,,\}, to be equal to the right hand side of (4.1) for all allowed values of gg and pp

1+μμ(ql()a()yl()a())p(qy1)2l()+2a()(1)p|μ|\displaystyle 1+\sum_{\mu\neq\emptyset}\prod_{\square\in\mu}(q^{l(\square)-a(\square)}y^{-l(\square)-a(\square)})^{p}(qy^{-1})^{2l(\square)+2a(\square)}(-1)^{p|\mu|}
×(1yl()a()qa()l()1)2g(1yl()a()1qa()l()1)(1yl()a()+1qa()l()1)Q|μ|\displaystyle\times\frac{(1-y^{l(\square)-a(\square)}q^{-a(\square)-l(\square)-1})^{2g}}{(1-y^{l(\square)-a(\square)-1}q^{-a(\square)-l(\square)-1})(1-y^{l(\square)-a(\square)+1}q^{-a(\square)-l(\square)-1})}Q^{|\mu|}
=μ(1)p|μ|q(g1p)κ(μ)(μ(qa()+l()2qa()+l()2)2g2)Q|μ|\displaystyle=\sum_{\mu}(-1)^{p|\mu|}q^{-(g-1-p)\kappa(\mu)}(\prod_{\square\in\mu}(q^{\frac{a(\square)+l(\square)}{2}}-q^{-\frac{a(\square)+l(\square)}{2}})^{2g-2})Q^{|\mu|} (74)

where κ(μ)=μ(i()j())\kappa(\mu)=\sum_{\square\in\mu}(i(\square)-j(\square)).
For y=1y=1 it was indicated that the following identity is crucial for proving the last conjectural equality

μ(l()a())=(j()i())\displaystyle\sum_{\square\in\mu}(l(\square)-a(\square))=\sum(j(\square)-i(\square)) (75)
𝒩(1,1,,1){\mathcal{N}(1,1,...,1)}(2)r{(\mathbb{C}^{2})^{r}}~r{\tilde{\mathcal{H}}^{r}}(2)r{(\mathbb{C}^{2})^{r}}r{\mathcal{H}^{r}}Sr(2) η  {S^{r}(\mathbb{C}^{2}){}{{}}{}{{}{}{}{{}}{{{{}{}{}{}}}{{{}{}{}{}}}{{}{}{}{}}}}{}{}{}{}{}{}\pgfsys@beginscope\pgfsys@setlinewidth{0.8pt}{}{}{}{}{{}}\pgfsys@moveto{-88.20372pt}{56.90552pt}\pgfsys@curveto{-103.91794pt}{56.90552pt}{-116.65648pt}{44.16698pt}{-116.65648pt}{28.45276pt}\pgfsys@curveto{-116.65648pt}{18.28748pt}{-111.23344pt}{8.89453pt}{-102.4301pt}{3.81189pt}\pgfsys@curveto{-95.14673pt}{-0.39333pt}{-86.37506pt}{-1.16048pt}{-79.22389pt}{1.4422pt}\pgfsys@stroke\pgfsys@invoke{}{{}{{}}{}{}{{}}{{{}}{{{}}{\pgfsys@beginscope\pgfsys@transformcm{0.9397}{0.342}{-0.342}{0.9397}{-78.84802pt}{1.579pt}\pgfsys@invoke{\lxSVG@closescope}\pgfsys@invoke{\lxSVG@closescope}\pgfsys@endscope}}{{}}}}\pgfsys@invoke{\lxSVG@closescope}\pgfsys@endscope{{}}\hbox{\hbox{{\pgfsys@beginscope{{}{}{{{}{}}}{{}{}} {{}{{}}}{{}{}}{}{{}{}} { }{{{{}}\pgfsys@beginscope\pgfsys@transformcm{1.0}{0.0}{0.0}{1.0}{-132.30565pt}{17.41684pt}\hbox{{\definecolor{pgfstrokecolor}{rgb}{0,0,0}\hbox{$${$\eta$}$$} }}\pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}} \pgfsys@invoke{\lxSVG@closescope }\pgfsys@endscope}}}}ρ\scriptstyle{\rho}Identity\scriptstyle{Identity}π\scriptstyle{\pi}
Figure 8:

4.2 Refined topological strings

The main purpose of this work is to give the generalisations of these conjectural identities for the refined topological string case.

Z\displaystyle Z (q1,q2,ρa,y,Q,𝐱)5d,instanton=𝐤𝐐𝐤|ν¯|=𝐤𝐚,𝐛=𝟏𝐫{}^{5d,instanton}(q_{1},q_{2},\rho_{a},y,Q,\bf{x})=\sum_{k}Q^{k}\sum_{|\underline{\nu}|=k}\prod_{a,b=1}^{r}
×\displaystyle\times (i,j)νa(1yρaρb1q1(νb)jtq2jνia1)(i,j)νb(1yρaρb1q1(νa)jtq2jνibj)(i,j)νa(1ρaρb1q1(νb)jtq2jνia1)(i,j)νb(1ρaρb1q1(νa)jtq2jνibj)\displaystyle\frac{\prod_{(i,j)\in\nu^{a}}(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{(\nu^{b})^{t}_{j}}q_{2}^{j-\nu_{i}^{a}-1})\prod_{(i,j)\in\nu^{b}}(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{(\nu^{a})^{t}_{j}}q_{2}^{j-\nu_{i}^{b}-j})}{\prod_{(i,j)\in\nu^{a}}(1-\rho_{a}\rho_{b}^{-1}q_{1}^{(\nu^{b})^{t}_{j}}q_{2}^{j-\nu_{i}^{a}-1})\prod_{(i,j)\in\nu^{b}}(1-\rho_{a}\rho_{b}^{-1}q_{1}^{(\nu^{a})^{t}_{j}}q_{2}^{j-\nu_{i}^{b}-j})}
×\displaystyle\times (μ¯,ν¯)|μ¯|=|ν¯|+d[(a=1ri=1cas=1μiaνiaρaq11iq2νias+1)\displaystyle\sum_{\begin{subarray}{c}(\underline{\mu},\underline{\nu})\\ |\underline{\mu}|=|\underline{\nu}|+d\end{subarray}}\Bigg{[}(\prod_{a=1}^{r}\prod_{i=1}^{c^{a}}\prod_{s=1}^{\mu^{a}_{i}-\nu^{a}_{i}}\rho_{a}q_{1}^{1-i}q_{2}^{-\nu^{a}_{i}-s+1})
×\displaystyle\times a,b=1ri=2ea+1j=1cbs=1μjbνjb(1yρaρb1q1jiq2νjb+sμia1)a,b=1ri=2ea+1j=1cbs=1μjbνjb(1ρaρb1q1jiq2νjb+sμia1)\displaystyle\frac{\prod_{a,b=1}^{r}\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu_{j}^{b}-\nu_{j}^{b}}(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu^{b}_{j}+s-\mu^{a}_{i}-1})}{\prod_{a,b=1}^{r}\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu_{j}^{b}-\nu_{j}^{b}}(1-\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu^{b}_{j}+s-\mu^{a}_{i}-1})}
×\displaystyle\times a,b=1ri=2ea+1j=1cbs=1μjbνjb(1ρaρb1q1jiq2νjb+sνi1a1)a,b=1ri=2ea+1j=1cbs=1μjbνjb(1yρaρb1q1jiq2νjb+sνi1a1)\displaystyle\frac{\prod_{a,b=1}^{r}\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu^{b}_{j}}(1-\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu^{b}_{j}+s-\nu^{a}_{i-1}-1})}{\prod_{a,b=1}^{r}\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu^{b}_{j}}(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu^{b}_{j}+s-\nu^{a}_{i-1}-1})}
×\displaystyle\times a,b=1rj=1cbs=1μjbνjb(1yρaρb1q1j1q2νjb+sμ1a1)a,b=1rj=1cbs=1μjbνjb(1ρaρb1q1j1q2νjb+sμ1a1)]H~μ¯(q1,q2,𝐱)\displaystyle\frac{\prod_{a,b=1}^{r}\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu^{b}_{j}}(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{j-1}q_{2}^{-\nu^{b}_{j}+s-\mu^{a}_{1}-1})}{\prod_{a,b=1}^{r}\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu^{b}_{j}}(1-\rho_{a}\rho_{b}^{-1}q_{1}^{j-1}q_{2}^{-\nu^{b}_{j}+s-\mu^{a}_{1}-1})}\Bigg{]}\tilde{H}_{\underline{\mu}}(q_{1},q_{2},\bf{x})

To substantiate the conjecture (4.1) for the refined topological strings in the rank r=1,2r=1,2 cases we have to find a map between the chemical potentials from the two sides. A natural generalisation for spacetime equivariant parameters is

q1\displaystyle q_{1} =\displaystyle= ty1,q2=q1y1,Q2=Ttqy1\displaystyle ty^{-1},\quad q_{2}=q^{-1}y^{-1},\quad Q_{2}=T\sqrt{\frac{t}{q}}y^{-1}
Q1\displaystyle Q_{1} =\displaystyle= tqy,Qfi=ρiρi+11\displaystyle\sqrt{\frac{t}{q}}y,\quad Q_{f_{i}}=\rho_{i}\rho_{i+1}^{-1} (77)

Making this change of variables conjecturally identifies the 5d5d Nekrasov partition function with the refined open topological string partition function

Zref,open(q,t,Q1,Q2,Qfi;x)=Z5d,instanton(q1,q2,ρa,y,T;x)|{q1=ty1,q2=q1y1,Q2=Ttqy1Q1=tqy,Qfi=ρiρi+11}\displaystyle Z^{ref,open}(q,t,Q_{1},Q_{2},Q_{f_{i}};\mbox{\bf{x}})=Z^{5d,instanton}(q_{1},q_{2},\rho_{a},y,T;\mbox{\bf{x}})\Bigg{\rvert}_{\{q_{1}=ty^{-1},\quad q_{2}=q^{-1}y^{-1},\quad Q_{2}=T\sqrt{\frac{t}{q}}y^{-1}Q_{1}=\sqrt{\frac{t}{q}}y,\quad Q_{f_{i}}=\rho_{i}\rho_{i+1}^{-1}\}} (78)

4.3 Lagrangian branes along the un-preferred vs preferred direction:Schur polynomials vs Macdonald polynomials

The Schur polynomials and the Macdonald polynomials are two of the symmetric functions bases in which we can expand he refined open topological string partition functions. The choice of the Schur polynomials corresponds Kozcaz:2018ndf to the lagrangian branes present on the unpreferred leg of the toric diagram, whereas the Macdonald polynomials basis corresponds to the lagrangian brane on the preferred leg of the toric diagram. Interestingly the modified Macdonald polynomials can be expanded in terms of both the Schur functions and the Macdonald polynomials as follows Chuang:2013wpa ; Haglund_2004 ; Haglund_2004

H~μ(x;q,t)\displaystyle\tilde{H}_{\mu}(\mbox{\bf{x}};q,t) =\displaystyle= λK~λμ(q,t)sλ(x)\displaystyle\sum_{\lambda}\tilde{K}_{\lambda\mu}(q,t)s_{\lambda}(\mbox{\bf{x}})
H~μ(x;q,t)\displaystyle\tilde{H}_{\mu}(\mbox{\bf{x}};q,t) =\displaystyle= ti(μi(i1))Jμ[x1t1;q,t1]\displaystyle t^{-\sum_{i}(\mu_{i}(i-1))}J_{\mu}\big{[}\frac{\mbox{\bf{x}}}{1-t^{-1}};q,t^{-1}\big{]} (79)
=\displaystyle= ti(μi(i1))s(λ)(1qa(s)tl(s)+1)Pλ(x;q,t)\displaystyle t^{-\sum_{i}(\mu_{i}(i-1))}\prod_{s\in(\lambda)}(1-q^{a(s)}t^{l(s)+1})P_{\lambda}(\mbox{\bf{x}};q,t)

where K~λμ(q,t)\tilde{K}_{\lambda\mu}(q,t) are the modified Kostka coefficients and have interesting combinatorial properties, ss specifies a box in the Young diagram, a(s)a(s) and l(s)l(s) denote the arm length and the leg length of the square ss repsectively, Jλ(x;q,t)J_{\lambda}(\mbox{\bf{x}};q,t) is defined as the integral form of the Macdonald polynomials

Jλ(x;q,t):=sD(λ)(1qa(s))tl(s)+1)Pλ(x;q,t)\displaystyle J_{\lambda}(\mbox{\bf{x}};q,t):=\prod_{s\in D(\lambda)}(1-q^{a(s)})t^{l(s)+1})P_{\lambda}(\mbox{\bf{x}};q,t) (80)

and

Jμ[x1t1;q,t1]:=Jμ[x1,x2,,t1x1,t1x2,,t2x1,t2x2,;q,t1].\displaystyle J_{\mu}\big{[}\frac{\mbox{\bf{x}}}{1-t^{-1}};q,t^{-1}\big{]}:=J_{\mu}\big{[}x_{1},x_{2},...,t^{-1}x_{1},t^{-1}x_{2},...,t^{-2}x_{1},t^{-2}x_{2},...;q,t^{-1}\big{]}. (81)

A crucial point is that the choice of the preferred or un preferred direction for the placement of the lagrangian branes corresponds to the expansion of the modified Macdonald polynomials in the Macdonald polynomials basis or the Schur functions basis.

5 Elliptic genus: a speculation for the refined open string invariants of special lagrangian branes for fully compactified web

In the same vein as the purported equality of the Donaldson-Thomas partition function of the CY3-fold and the K-theory partition function of the framed quiver moduli space given in the last section, we give an expression for the generating function of the elliptic genus of the same framed moduli space of section 2 and propose that

TheDonaldsonThomaspartitionfunctionsoftheCY3folds(fig.10)\displaystyle The\quad Donaldson-Thomas\quad partition\quad functions\quad of\quad the\quad CY3-folds(fig.\ref{fullcompAnfibP1})
=\displaystyle=
Thegeneratingfunctionoftheellipticgenusgivenin(5,5)\displaystyle The\quad generating\quad function\quad of\quad the\quad elliptic\quad genus\quad given\quad in\quad(\ref{eq:GEG1},\ref{eq:GEG2})

In the presence of vector bundles on quiver moduli space 𝒩(r,k+d,d)\mathcal{N}(r,k+d,d), the natural generalisation of the χy\chi_{y}-genus is the so-called elliptic genus. The elliptic genus contains topological information about the vector bundles and can be arranged as a generating series of cohomology groups of the vector bundles. To define it, consider a vector bundle VV on XX and define the formal product

(V)=n1(ΛyqnVΛy1qnSqn(VV))\displaystyle\mathcal{E}(V)=\otimes_{n\geq 1}(\Lambda_{-yq^{n}}V^{\vee}\otimes\Lambda_{-y^{-1}q^{n}}\otimes S_{q^{n}}(V\oplus V^{\vee})) (83)

It is interesting to note that this formal product is the vector bundle analogue of the Jacobi triple product formulaGritsenko:1999nm ; Haghighat:2013gba . The elliptic genus χelliptic(X,V;y,q)\chi_{elliptic}(X,V;y,q) is defined by its relation to the chi-y genus as

χelliptic(X,V;y,q):=yrkTX/2χy(X,(TX)V)\displaystyle\chi_{elliptic}(X,V;y,q):=y^{-rkT_{X}/2}\chi_{-y}(X,\mathcal{E}(T_{X})\otimes V) (84)

Then using Riemann-Roch theorem the elliptic genus χelliptic(X,V;y,q)\chi_{elliptic}(X,V;y,q) can be expressed by

χelliptic(X,V;y,q)\displaystyle\chi_{elliptic}(X,V;y,q) =\displaystyle= [X]yrkTX/2ch(ΛyTX)ch((TX))td(TX)ch(V)\displaystyle\int_{[X]}y^{-rkT_{X}/2}\mbox{ch}(\Lambda_{-y}T_{X}^{\vee})\mbox{ch}(\mathcal{E}(T_{X}))\mbox{td}(T_{X})\mbox{ch}(V) (85)
=\displaystyle= X((l=1devk))i=1nxiθ(xi2πiz,τ)θ(xi2πi,τ)i=1mθ(ui2πi,τ)θ(ui2πiz,τ)\displaystyle\int_{X}((\sum_{l=1}^{d}e^{v_{k}}))\prod_{i=1}^{n}x_{i}\frac{\theta(\frac{x_{i}}{2\pi i}-z,\tau)}{\theta(\frac{x_{i}}{2\pi i},\tau)}\prod_{i=1}^{m}\frac{\theta(\frac{u_{i}}{2\pi i},\tau)}{\theta(\frac{u_{i}}{2\pi i}-z,\tau)}

where q=e2πiτ,y=e2πizq=e^{2\pi i\tau},y=e^{2\pi iz} and θ(z,τ)=q1/8y1/2y1/2il=1(1ql)(1qly)(1qly1)\theta(z,\tau)=q^{1/8}\frac{y^{1/2}-y^{-1/2}}{i}\prod_{l=1}(1-q^{l})(1-q^{l}y)(1-q^{l}y^{-1}). For the moduli space 𝒩(r,n+d,d)\mathcal{N}(r,n+d,d) one has to generalise the above definitions to include virtual schemes allowing an equivariant torus action and use equivariant localisation. We follow the fixed point formulas given in these references to write down the final expressions for χy\chi_{y} genus and elliptic genus of 𝒩(r,n+d,d)\mathcal{N}(r,n+d,d). Note that TXT_{X} for X=𝒩(r,n+d,d)X=\mathcal{N}(r,n+d,d) is given in (2). For a quick review of the fixed point formulae see appendix (D).
The computation of the refined open topological string amplitude corresponding to the Euler characteristic and χy\chi_{y} genus was relatively simple in the sense that the lagrangian brane was put on the external leg of the toric diagram. So it was a topological amplitude in the presence of the external branes. In the case of a totally compactified web diagram, all the legs are essentially internal. The instanton partition function of the six dimensional theory with surface defect can be interpreted as the generating function of the elliptic genus

Zr,d6d,quiver=kQρkχell(𝒩(r,k+d,d),q1,q2,y,ρa,Qσ)\displaystyle Z^{6d,quiver}_{r,d}=\sum_{k}Q_{\rho}^{k}\chi_{ell}(\mathcal{N}(r,k+d,d),q_{1},q_{2},y,\rho_{a},Q_{\sigma}) (86)

Q2Q_{2}Q1Q_{1}
Figure 9: totally compactified toric web of the the total space of the bundle 𝒪(1)𝒪(1)1\mathcal{O}(-1)\oplus\mathcal{O}(-1)\to\mathbb{P}^{1}

Zr,d6d,quiverZ^{6d,quiver}_{r,d} will provide the prediction for Z~open,dref\tilde{Z}^{ref}_{open,d} for general rank rr and a fully compactified web. In figure 9 we give the totally compactified toric web of the the total space of the bundle 𝒪(1)𝒪(1)1\mathcal{O}(-1)\oplus\mathcal{O}(-1)\to\mathbb{P}^{1}.

Z1,d6d,quiver=\displaystyle Z^{6d,quiver}_{1,d}= kQρkχell(𝒩(1,k+d,d),q1,q2,y,ρ1,Qσ)=|ν¯|=kQρk×\displaystyle\sum_{k}Q_{\rho}^{k}\chi_{ell}(\mathcal{N}(1,k+d,d),q_{1},q_{2},y,\rho_{1},Q_{\sigma})=\sum_{|\underline{\nu}|=k}Q_{\rho}^{k}\times
(i,j)ν(1yq1(ν)jtq2jνi1)(1q1(ν)jtq2jνi1)k=1(1yQσkq1(ν1)jtq2jνi1)(1y1Qσkq1(ν)jtq2j+νi+1)(1Qσkq1(ν1)jtq2jνi1)(1Qσkq1(ν1)jtq2j+νi+1)\displaystyle\prod_{(i,j)\in\nu}\frac{(1-yq_{1}^{(\nu)^{t}_{j}}q_{2}^{j-\nu_{i}-1})}{(1-q_{1}^{(\nu)^{t}_{j}}q_{2}^{j-\nu_{i}-1})}\prod_{k=1}\frac{(1-yQ_{\sigma}^{k}q_{1}^{(\nu^{1})^{t}_{j}}q_{2}^{j-\nu_{i}-1})(1-y^{-1}Q_{\sigma}^{k}q_{1}^{-(\nu)^{t}_{j}}q_{2}^{-j+\nu_{i}+1})}{(1-Q_{\sigma}^{k}q_{1}^{(\nu^{1})^{t}_{j}}q_{2}^{j-\nu_{i}-1})(1-Q_{\sigma}^{k}q_{1}^{-(\nu^{1})^{t}_{j}}q_{2}^{-j+\nu_{i}+1})}
(i,j)ν(1yq1(ν)jtq2jνij)(1q1(ν)jtq2jνij)k=1(1yQσkq1(ν)jtq2jνij)(1y1Qσkq1(ν1)jtq2j+νi+j)(1Qσkq1(ν)jtq2jνij)(1Qσkq1(ν)jtq2j+νi+j)\displaystyle\prod_{(i,j)\in\nu}\frac{(1-yq_{1}^{(\nu)^{t}_{j}}q_{2}^{j-\nu_{i}-j})}{(1-q_{1}^{(\nu)^{t}_{j}}q_{2}^{j-\nu_{i}-j})}\prod_{k=1}\frac{(1-yQ_{\sigma}^{k}q_{1}^{(\nu)^{t}_{j}}q_{2}^{j-\nu_{i}-j})(1-y^{-1}Q_{\sigma}^{k}q_{1}^{-(\nu^{1})^{t}_{j}}q_{2}^{-j+\nu_{i}+j})}{(1-Q_{\sigma}^{k}q_{1}^{(\nu)^{t}_{j}}q_{2}^{j-\nu_{i}-j})(1-Q_{\sigma}^{k}q_{1}^{-(\nu)^{t}_{j}}q_{2}^{-j+\nu_{i}+j})}
(μ,ν)|μ|=|ν|+d[(i=1cs=1μiνiρ1q11iq2νis+1)\displaystyle\sum_{\begin{subarray}{c}(\mu,\nu)\\ |\mu|=|\nu|+d\end{subarray}}\Bigg{[}\bigg{(}\prod_{i=1}^{c}\prod_{s=1}^{\mu_{i}-\nu_{i}}\rho_{1}q_{1}^{1-i}q_{2}^{-\nu_{i}-s+1}\bigg{)}
×\displaystyle\times
(i=2e+1j=1cs=1μjνj\displaystyle\bigg{(}\prod_{i=2}^{e+1}\prod_{j=1}^{c}\prod_{s=1}^{\mu_{j}-\nu_{j}}
(1yq1jiq2νj+sμi1)(1q1jiq2νj+sμi1)k=1(1yQσkq1jiq2νj+sμi1)(1y1Qσkq1j+iq2νjs+μi+1)(1Qσkq1jiq2νj+sμi1)(1Qσkq1j+iq2νjs+μi+1))\displaystyle\frac{(1-yq_{1}^{j-i}q_{2}^{\nu_{j}+s-\mu_{i}-1})}{(1-q_{1}^{j-i}q_{2}^{\nu_{j}+s-\mu_{i}-1})}\prod_{k=1}\frac{(1-yQ_{\sigma}^{k}q_{1}^{j-i}q_{2}^{\nu_{j}+s-\mu_{i}-1})(1-y^{-1}Q_{\sigma}^{k}q_{1}^{-j+i}q_{2}^{-\nu_{j}-s+\mu_{i}+1})}{(1-Q_{\sigma}^{k}q_{1}^{j-i}q_{2}^{\nu_{j}+s-\mu_{i}-1})(1-Q_{\sigma}^{k}q_{1}^{-j+i}q_{2}^{-\nu_{j}-s+\mu_{i}+1})}\bigg{)}
(i=2e+1j=1cp=0(1yq1jiq2νjνi1p1)(1yq1jiq2μjνi1+p)i=2e+1j=1cp=0(1q1jiq2νjνi1p1)(1q1jiq2μjνi1+p)\displaystyle\bigg{(}\frac{\prod_{i=2}^{e+1}\prod_{j=1}^{c}\prod_{p=0}(1-yq_{1}^{j-i}q_{2}^{\nu_{j}-\nu_{i-1}-p-1})(1-yq_{1}^{j-i}q_{2}^{\mu_{j}-\nu_{i-1}+p})}{\prod_{i=2}^{e+1}\prod_{j=1}^{c}\prod_{p=0}(1-q_{1}^{j-i}q_{2}^{\nu_{j}-\nu_{i-1}-p-1})(1-q_{1}^{j-i}q_{2}^{\mu_{j}-\nu_{i-1}+p})}
i=2e+1j=1cp=0(1yQσkq1jiq2νjνi1p1)(1yQσkq1jiq2μjνi1+p)i=2e+1j=1cp=0(1Qσkq1jiq2νjνi1p1)(1Qσkq1jiq2μjνi1+p)\displaystyle\frac{\prod_{i=2}^{e+1}\prod_{j=1}^{c}\prod_{p=0}(1-yQ_{\sigma}^{k}q_{1}^{j-i}q_{2}^{\nu_{j}-\nu_{i-1}-p-1})(1-yQ_{\sigma}^{k}q_{1}^{j-i}q_{2}^{\mu_{j}-\nu_{i-1}+p})}{\prod_{i=2}^{e+1}\prod_{j=1}^{c}\prod_{p=0}(1-Q_{\sigma}^{k}q_{1}^{j-i}q_{2}^{\nu_{j}-\nu_{i-1}-p-1})(1-Q_{\sigma}^{k}q_{1}^{j-i}q_{2}^{\mu_{j}-\nu_{i-1}+p})}
i=2e+1j=1cp=0(1y1Qσkq1j+iq2νj+νi1+p+1)(1y1Qσkq1j+iq2μj+νi1p)i=2e+1j=1cp=0(1Qσkq1j+iq2νj+νi1+p+1)(1Qσkq1j+iq2μj+νi1p))\displaystyle\frac{\prod_{i=2}^{e+1}\prod_{j=1}^{c}\prod_{p=0}(1-y^{-1}Q_{\sigma}^{k}q_{1}^{-j+i}q_{2}^{-\nu_{j}+\nu_{i-1}+p+1})(1-y^{-1}Q_{\sigma}^{k}q_{1}^{-j+i}q_{2}^{-\mu_{j}+\nu_{i-1}-p})}{\prod_{i=2}^{e+1}\prod_{j=1}^{c}\prod_{p=0}(1-Q_{\sigma}^{k}q_{1}^{-j+i}q_{2}^{-\nu_{j}+\nu_{i-1}+p+1})(1-Q_{\sigma}^{k}q_{1}^{-j+i}q_{2}^{-\mu_{j}+\nu_{i-1}-p})}\bigg{)}
(i=2e+1j=1cs=1μjνj\displaystyle\bigg{(}\prod_{i=2}^{e+1}\prod_{j=1}^{c}\prod_{s=1}^{\mu_{j}-\nu_{j}}
(1yq1j1q2νj+sμ11)(1q1j1q2νj+sμ11)k=1(1yQσkq1j1q2νj+sμ11)(1y1Qσkq1j+1q2νjs+μ1+1)(1Qσkq1j1q2νj+sμ11)(1Qσkq1j+1q2νjs+μ1+1))]\displaystyle\frac{(1-yq_{1}^{j-1}q_{2}^{\nu_{j}+s-\mu_{1}-1})}{(1-q_{1}^{j-1}q_{2}^{\nu_{j}+s-\mu_{1}-1})}\prod_{k=1}\frac{(1-yQ_{\sigma}^{k}q_{1}^{j-1}q_{2}^{\nu_{j}+s-\mu_{1}-1})(1-y^{-1}Q_{\sigma}^{k}q_{1}^{-j+1}q_{2}^{-\nu_{j}-s+\mu_{1}+1})}{(1-Q_{\sigma}^{k}q_{1}^{j-1}q_{2}^{\nu_{j}+s-\mu_{1}-1})(1-Q_{\sigma}^{k}q_{1}^{-j+1}q_{2}^{-\nu_{j}-s+\mu_{1}+1})}\bigg{)}\Bigg{]}

For the values of r2r\geq 2 the compactified web diagram is given in the figure (10), Iqbal:2015dra . This figure is the web diagram of a CY3-fold described as a resolved Ar1A_{r-1} fibration over 1\mathbb{P}^{1}. We can also see the diagram as obtained by gluing r1r-1 Hirzebruch surfaces. The local geometry of the intersection between i1i-1-th and ii-th Hirzebruch surfaces is given by the bundle 𝒪(r+2i+2)𝒪(r2i)1\mathcal{O}(-r+2i+2)\oplus\mathcal{O}(r-2i)\to\mathbb{P}^{1} for i=1,2,,ri=1,2,...,r.

\displaystyle\hskip 199.16928pt\vdots
\displaystyle\hskip 199.16928pt\vdots
\displaystyle\hskip 199.16928pt\vdots
Figure 10: fully compactified web diagram of the the total space of Ar1A_{r-1} fibration over 1\mathbb{P}^{1}

The corresponding elliptic genus for the quiver moduli 𝒩(r,k+d,d)\mathcal{N}(r,k+d,d) for r2r\geq 2 is given by

Zr2,d6d,quiver=\displaystyle Z^{6d,quiver}_{r\geq 2,d}=
kQρkχell(𝒩(r,k+d,d),q1,q2,y,ρa,Qσ)=|ν¯|=kQρk×\displaystyle\sum_{k}Q_{\rho}^{k}\chi_{ell}(\mathcal{N}(r,k+d,d),q_{1},q_{2},y,\rho_{a},Q_{\sigma})=\sum_{|\underline{\nu}|=k}Q_{\rho}^{k}\times
a,b=1r(i,j)νa(1yρaρb1q1(νb)jtq2jνia1)(1ρaρb1q1(νb)jtq2jνia1)k=1(1yQσkρaρb1q1(νb)jtq2jνia1)(1y1Qσkρa1ρbq1(νb)jtq2j+νia+1)(1Qσkρaρb1q1(νb)jtq2jνia1)(1Qσkρa1ρbq1(νb)jtq2j+νia+1)\displaystyle\prod_{a,b=1}^{r}\prod_{(i,j)\in\nu^{a}}\frac{(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{(\nu^{b})^{t}_{j}}q_{2}^{j-\nu_{i}^{a}-1})}{(1-\rho_{a}\rho_{b}^{-1}q_{1}^{(\nu^{b})^{t}_{j}}q_{2}^{j-\nu_{i}^{a}-1})}\prod_{k=1}\frac{(1-yQ_{\sigma}^{k}\rho_{a}\rho_{b}^{-1}q_{1}^{(\nu^{b})^{t}_{j}}q_{2}^{j-\nu_{i}^{a}-1})(1-y^{-1}Q_{\sigma}^{k}\rho_{a}^{-1}\rho_{b}q_{1}^{-(\nu^{b})^{t}_{j}}q_{2}^{-j+\nu_{i}^{a}+1})}{(1-Q_{\sigma}^{k}\rho_{a}\rho_{b}^{-1}q_{1}^{(\nu^{b})^{t}_{j}}q_{2}^{j-\nu_{i}^{a}-1})(1-Q_{\sigma}^{k}\rho_{a}^{-1}\rho_{b}q_{1}^{-(\nu^{b})^{t}_{j}}q_{2}^{-j+\nu_{i}^{a}+1})}
a,b=1r(i,j)νa(1yρaρb1q1(νa)jtq2jνibj)(1ρaρb1q1(νa)jtq2jνibj)k=1(1yQσkρaρb1q1(νa)jtq2jνibj)(1y1Qσkρa1ρbq1(νa)jtq2j+νib+j)(1Qσkρaρb1q1(νa)jtq2jνibj)(1Qσkρa1ρbq1(νa)jtq2j+νib+j)\displaystyle\prod_{a,b=1}^{r}\prod_{(i,j)\in\nu^{a}}\frac{(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{(\nu^{a})^{t}_{j}}q_{2}^{j-\nu_{i}^{b}-j})}{(1-\rho_{a}\rho_{b}^{-1}q_{1}^{(\nu^{a})^{t}_{j}}q_{2}^{j-\nu_{i}^{b}-j})}\prod_{k=1}\frac{(1-yQ_{\sigma}^{k}\rho_{a}\rho_{b}^{-1}q_{1}^{(\nu^{a})^{t}_{j}}q_{2}^{j-\nu_{i}^{b}-j})(1-y^{-1}Q_{\sigma}^{k}\rho_{a}^{-1}\rho_{b}q_{1}^{-(\nu^{a})^{t}_{j}}q_{2}^{-j+\nu_{i}^{b}+j})}{(1-Q_{\sigma}^{k}\rho_{a}\rho_{b}^{-1}q_{1}^{(\nu^{a})^{t}_{j}}q_{2}^{j-\nu_{i}^{b}-j})(1-Q_{\sigma}^{k}\rho_{a}^{-1}\rho_{b}q_{1}^{-(\nu^{a})^{t}_{j}}q_{2}^{-j+\nu_{i}^{b}+j})}
(μ,ν)|μ|=|ν|+d[(a=1ri=1cas=1μiaνiaρaqa1iq2νias+1)\displaystyle\sum_{\begin{subarray}{c}(\mu,\nu)\\ |\mu|=|\nu|+d\end{subarray}}\Bigg{[}\bigg{(}\prod_{a=1}^{r}\prod_{i=1}^{c^{a}}\prod_{s=1}^{\mu^{a}_{i}-\nu^{a}_{i}}\rho_{a}q_{a}^{1-i}q_{2}^{-\nu^{a}_{i}-s+1}\bigg{)}
×\displaystyle\times
a,b=1r(i=2ea+1j=1cbs=1μjbνjb\displaystyle\prod_{a,b=1}^{r}\bigg{(}\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu_{j}^{b}}
(1yρaρb1q1jiq2νjb+sμia1)(1ρaρb1q1jiq2νjb+sμia1)k=1(1yQσkρaρb1q1jiq2νjb+sμia1)(1y1Qσkρa1ρbq1j+iq2νjbs+μia+1)(1Qσkρaρb1q1jiq2νjb+sμia1)(1Qσkρa1ρbq1j+iq2νjbs+μia+1))\displaystyle\frac{(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu_{j}^{b}+s-\mu_{i}^{a}-1})}{(1-\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu_{j}^{b}+s-\mu_{i}^{a}-1})}\prod_{k=1}\frac{(1-yQ_{\sigma}^{k}\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu_{j}^{b}+s-\mu_{i}^{a}-1})(1-y^{-1}Q_{\sigma}^{k}\rho_{a}^{-1}\rho_{b}q_{1}^{-j+i}q_{2}^{-\nu_{j}^{b}-s+\mu_{i}^{a}+1})}{(1-Q_{\sigma}^{k}\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu_{j}^{b}+s-\mu_{i}^{a}-1})(1-Q_{\sigma}^{k}\rho_{a}^{-1}\rho_{b}q_{1}^{-j+i}q_{2}^{-\nu_{j}^{b}-s+\mu_{i}^{a}+1})}\bigg{)}
a,b=1r(i=2ea+1j=1cbp=0(1yρaρb1q1jiq2νjbνi1ap1)(1yρaρb1q1jiq2μjbνi1a+p)i=2ea+1j=1cbp=0(1ρaρb1q1jiq2νjbνi1ap1)(1ρaρb1q1jiq2μjbνi1a+p)\displaystyle\prod_{a,b=1}^{r}\bigg{(}\frac{\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{p=0}(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu_{j}^{b}-\nu_{i-1}^{a}-p-1})(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\mu_{j}^{b}-\nu_{i-1}^{a}+p})}{\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{p=0}(1-\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu_{j}^{b}-\nu_{i-1}^{a}-p-1})(1-\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\mu_{j}^{b}-\nu_{i-1}^{a}+p})}
i=2ea+1j=1cbp=0(1yQσkρaρb1q1jiq2νjbνi1ap1)(1yQσkρaρb1q1jiq2μjbνi1a+p)i=2ea+1j=1cbp=0(1Qσkρaρb1q1jiq2νjbνi1ap1)(1Qσkρaρb1q1jiq2μjbνi1a+p)\displaystyle\frac{\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{p=0}(1-yQ_{\sigma}^{k}\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu_{j}^{b}-\nu_{i-1}^{a}-p-1})(1-yQ_{\sigma}^{k}\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\mu_{j}^{b}-\nu_{i-1}^{a}+p})}{\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{p=0}(1-Q_{\sigma}^{k}\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\nu_{j}^{b}-\nu_{i-1}^{a}-p-1})(1-Q_{\sigma}^{k}\rho_{a}\rho_{b}^{-1}q_{1}^{j-i}q_{2}^{\mu_{j}^{b}-\nu_{i-1}^{a}+p})}
i=2ea+1j=1cbp=0(1y1Qσkρbρa1q1j+iq2νjb+νi1a+p+1)(1y1Qσkρbρa1q1j+iq2μjb+νi1ap)i=2ea+1j=1cbp=0(1Qσkρbρa1q1j+iq2νjb+νi1a+p+1)(1Qσkρbρa1q1j+iq2μjb+νi1ap))\displaystyle\frac{\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{p=0}(1-y^{-1}Q_{\sigma}^{k}\rho_{b}\rho_{a}^{-1}q_{1}^{-j+i}q_{2}^{-\nu_{j}^{b}+\nu_{i-1}^{a}+p+1})(1-y^{-1}Q_{\sigma}^{k}\rho_{b}\rho_{a}^{-1}q_{1}^{-j+i}q_{2}^{-\mu_{j}^{b}+\nu_{i-1}^{a}-p})}{\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{p=0}(1-Q_{\sigma}^{k}\rho_{b}\rho_{a}^{-1}q_{1}^{-j+i}q_{2}^{-\nu_{j}^{b}+\nu_{i-1}^{a}+p+1})(1-Q_{\sigma}^{k}\rho_{b}\rho_{a}^{-1}q_{1}^{-j+i}q_{2}^{-\mu_{j}^{b}+\nu_{i-1}^{a}-p})}\bigg{)}
(a,b=1ri=2ea+1j=1cbs=1μjbνjb\displaystyle\bigg{(}\prod_{a,b=1}^{r}\prod_{i=2}^{e^{a}+1}\prod_{j=1}^{c^{b}}\prod_{s=1}^{\mu^{b}_{j}-\nu_{j}^{b}}
(1yρaρb1q1j1q2νjb+sμ1a1)(1ρaρb1q1j1q2νjb+sμ1a1)k=1(1yQσkρaρb1q1j1q2νjb+sμ1a1)(1y1Qσkρa1ρbq1j+1q2νjbs+μ1a+1)(1Qσkρaρb1q1j1q2νjb+sμ1a1)(1Qσkρa1ρbq1j+1q2νjbs+μ1a+1))]\displaystyle\frac{(1-y\rho_{a}\rho_{b}^{-1}q_{1}^{j-1}q_{2}^{\nu_{j}^{b}+s-\mu_{1}^{a}-1})}{(1-\rho_{a}\rho_{b}^{-1}q_{1}^{j-1}q_{2}^{\nu_{j}^{b}+s-\mu_{1}^{a}-1})}\prod_{k=1}\frac{(1-yQ_{\sigma}^{k}\rho_{a}\rho_{b}^{-1}q_{1}^{j-1}q_{2}^{\nu_{j}^{b}+s-\mu_{1}^{a}-1})(1-y^{-1}Q_{\sigma}^{k}\rho_{a}^{-1}\rho_{b}q_{1}^{-j+1}q_{2}^{-\nu_{j}^{b}-s+\mu_{1}^{a}+1})}{(1-Q_{\sigma}^{k}\rho_{a}\rho_{b}^{-1}q_{1}^{j-1}q_{2}^{\nu_{j}^{b}+s-\mu_{1}^{a}-1})(1-Q_{\sigma}^{k}\rho_{a}^{-1}\rho_{b}q_{1}^{-j+1}q_{2}^{-\nu_{j}^{b}-s+\mu_{1}^{a}+1})}\bigg{)}\Bigg{]}

6 Conclusions

We gave expressions for the χy\chi_{y} genus and elliptic genus for a quiver moduli space 𝒩(r,n+d,d)\mathcal{N}(r,n+d,d) described by stable representations of an enhanced ADHM quiver of type (n+d,d,r). Then a conjecture is formulated that equates generating function of χy\chi_{y} genus for r=1r=1 and r=2r=2 with the open topological string partition functions on CY 3-folds given by partially compactified resolved conifold and partially compactified total space of the bundle 𝒪(2,2)\mathcal{O}(-2,-2) of 1×1\mathbb{P}^{1}\times\mathbb{P}^{1}. Finally it is suggested that the generating function of the elliptic genus may be related to the open topological string partition function on these CY 3-folds whose corresponding web diagrams are fully compactified. To discuss the conjectures for SU(3)SU(3) and higher rank 5d mass deformed gauge theories, it turns out to be necessary to deal with what are called shifted web diagrams Bastian:2018fba . This is a work in progress.

Acknowledgement

The author would like to thank Amer Iqbal for suggesting the problem and giving various comments. Moreover the support provided by Abdus Salam School of Mathematical Sciences is gratefully acknowledged.

Appendix A Zopenref(Qb,Qf,Qm,q,t;x)Z_{open}^{ref}(Q_{b},Q_{f},Q_{m},q,t;x) on the partially compactified geometry of section (3.4): Alternate expression

In section (3.4) we computed the topological string partition function on the flop of this geometry and then analytically continuted the partition function to the pre-flopped geometry. In this appendix we compute the partition function using the refined topological vertex without flopping the geometry. For this case the partially compactified toric diagram of the total space of the bundle 𝒪(2,2)\mathcal{O}(-2,-2) of 1×1\mathbb{P}^{1}\times\mathbb{P}^{1} is given below
λ\lambdaQmQ_{m}QfQ_{f}QbQ_{b}

The building block for r=2r=2 corresponding locally to 𝒪(2)𝒪(0)1\mathcal{O}(-2)\oplus\mathcal{O}(0)\to\mathbb{P}^{1} is given by

Zν1ν2,λ,ρt(q,t,Qf)=(Qf)|μ|Cλμν2t(q,t)Cμtρtν2t(q,t)fμ(t,q)\displaystyle Z_{\nu_{1}\nu_{2},\lambda,\rho^{t}}(q,t,Q_{f})=\sum(-Q_{f})^{|\mu|}C_{\lambda\mu\nu^{t}_{2}}(q,t)C_{\mu^{t}\rho^{t}\nu^{t}_{2}}(q,t)f_{\mu}(t,q) (89)

with framing factor fμ(t,q)f_{\mu}(t,q) and refined topological vertex Cλμν(t,q)C_{\lambda\mu\nu}(t,q) given by

fμ(t,q)\displaystyle f_{\mu}(t,q) =\displaystyle= (1)|μ|tηt/2|η|/2qη2/2+|η|/2\displaystyle(-1)^{|\mu|}t^{||\eta^{t}||/2-|\eta|/2}q^{-||\eta||^{2}/2+|\eta|/2}
Cλμν(t,q)\displaystyle C_{\lambda\mu\nu}(t,q) =\displaystyle= (tq)μ22qκ(μ)+ν22Z~ν(t,q)\displaystyle(\frac{t}{q})^{\frac{||\mu||^{2}}{2}}q^{\frac{\kappa(\mu)+||\nu||^{2}}{2}}\tilde{Z}_{\nu}(t,q) (90)
×\displaystyle\times η(qt)|η|+|λ||μ|2sλt/η(tρqν)sμ/η(tνtqρ)\displaystyle\sum_{\eta}(\frac{q}{t})^{\frac{|\eta|+|\lambda|-|\mu|}{2}}s_{\lambda^{t}/\eta}(t^{-\rho}q^{-\nu})s_{\mu/\eta}(t^{-\nu^{t}}q^{-\rho})

The gluing of the local geometries 𝒪(2)𝒪(0)1\mathcal{O}(-2)\oplus\mathcal{O}(0)\to\mathbb{P}^{1}, 𝒪(0)𝒪(2)1\mathcal{O}(0)\oplus\mathcal{O}(-2)\to\mathbb{P}^{1} taking into account the normal-to-the-base directions as well as the compactification of the external leg without brane corresponds to the following amplitude

Zopenref(Qf,Qb,Qm,q,t;x)\displaystyle Z_{open}^{ref}(Q_{f},Q_{b},Q_{m},q,t;x) =\displaystyle= (Qm)|ρ|(Qb)|ν1|+|ν2|f~ν1t(q,t)f~ν2(t,q)Zν1tν2t,,ρ(t,q,Qf)\displaystyle\sum(-Q_{m})^{|\rho|}(-Q_{b})^{|\nu_{1}|+|\nu_{2}|}\tilde{f}_{\nu_{1}^{t}}(q,t)\tilde{f}_{\nu_{2}}(t,q)Z_{\nu_{1}^{t}\nu_{2}^{t},\emptyset,\rho}(t,q,Q_{f})
×\displaystyle\times Zν1ν2,λ,ρt(q,t,Qf)\displaystyle Z_{\nu_{1}\nu_{2},\lambda,\rho^{t}}(q,t,Q_{f})

Using the following skew Schur function identities repeatedly

λsλ/α(x)sλ/β(y)\displaystyle\sum_{\lambda}s_{\lambda/\alpha}(x)s_{\lambda/\beta}(y) =\displaystyle= i,j(1xiyj)1sβ/η(x)sα/η(y)\displaystyle\prod_{i,j}(1-x_{i}y_{j})^{-1}\sum s_{\beta/\eta}(x)s_{\alpha/\eta}(y)
λsλt/α(x)sλ/β(y)\displaystyle\sum_{\lambda}s_{\lambda^{t}/\alpha}(x)s_{\lambda/\beta}(y) =\displaystyle= i,j(1+xiyj)sβt/ηt(x)sαt/η(y)\displaystyle\prod_{i,j}(1+x_{i}y_{j})\sum s_{\beta^{t}/\eta^{t}}(x)s_{\alpha^{t}/\eta}(y) (92)

we get the following expression

Zopenref(Qf,Qb,Qm,q,t;x)\displaystyle Z_{open}^{ref}(Q_{f},Q_{b},Q_{m},q,t;x) =\displaystyle= (Qb)|ν1|+|ν2|tν1t2+ν2t22qν12+ν222\displaystyle\sum(-Q_{b})^{|\nu_{1}|+|\nu_{2}|}t^{\frac{||\nu_{1}^{t}||^{2}+||\nu_{2}^{t}||^{2}}{2}}q^{\frac{||\nu_{1}||^{2}+||\nu_{2}||^{2}}{2}}
×\displaystyle\times f~ν1t(q,t)f~ν2(t,q)Z~ν1t(q,t)Z~ν2t(q,t)Z~ν1(t,q)Z~ν2(t,q)\displaystyle\tilde{f}_{\nu_{1}^{t}}(q,t)\tilde{f}_{\nu_{2}}(t,q)\tilde{Z}_{\nu_{1}^{t}}(q,t)\tilde{Z}_{\nu_{2}^{t}}(q,t)\tilde{Z}_{\nu_{1}}(t,q)\tilde{Z}_{\nu_{2}}(t,q)
×\displaystyle\times i,j(1Qmqν2i+j12ti12ν2jt)i,j(1+qi1tν1it+12xj)\displaystyle\prod_{i,j}(1-Q_{m}q^{-\nu_{2i}+j-\frac{1}{2}}t^{i-\frac{1}{2}-\nu_{2j}^{t}})\prod_{i,j}(1+q^{i-1}t^{-\nu_{1i}^{t}+\frac{1}{2}}x_{j})
×\displaystyle\times i,j(1Qfqi1ν2jtjν1it)1i,j(1Qfqjν1iti1ν2jt)1\displaystyle\prod_{i,j}(1-Q_{f}q^{i-1-\nu_{2j}}t^{j-\nu_{1i}^{t}})^{-1}\prod_{i,j}(1-Q_{f}q^{j-\nu_{1i}}t^{i-1-\nu_{2j}^{t}})^{-1}
×\displaystyle\times i,j(1QmQfqj12ν2iti12ν1jt)i,j(1+Qfqi1tν2it+12xj)\displaystyle\prod_{i,j}(1-Q_{m}Q_{f}q^{j-\frac{1}{2}-\nu_{2i}}t^{i-\frac{1}{2}-\nu_{1j}^{t}})\prod_{i,j}(1+Q_{f}q^{i-1}t^{-\nu_{2i}^{t}+\frac{1}{2}}x_{j})
×\displaystyle\times i,j(1QmQfqi12ν1jtj12ν2it)i,j(1QmQf2qj12ν1jti12ν1jt)\displaystyle\prod_{i,j}(1-Q_{m}Q_{f}q^{i-\frac{1}{2}-\nu_{1j}}t^{j-\frac{1}{2}-\nu_{2i}^{t}})\prod_{i,j}(1-Q_{m}Q_{f}^{2}q^{j-\frac{1}{2}-\nu_{1j}}t^{i-\frac{1}{2}-\nu_{1j}^{t}})
×\displaystyle\times i,j(1+QfQmtqtν2itqi12xj)1i,j(1+Qf2Qmtqqi12tν1itxj)1\displaystyle\prod_{i,j}(1+Q_{f}Q_{m}\frac{t}{q}t^{-\nu_{2i}^{t}}q^{i-\frac{1}{2}}x_{j})^{-1}\prod_{i,j}(1+Q_{f}^{2}Q_{m}\frac{t}{q}q^{i-\frac{1}{2}}t^{-\nu_{1i}^{t}}x_{j})^{-1}

The gauge theory instanton part of the partition function that is relevant for the comparison with quiver partition function is given by

Z~openref(Qf,Qb,Qm,q,t;x)\displaystyle\tilde{Z}_{open}^{ref}(Q_{f},Q_{b},Q_{m},q,t;x) =\displaystyle= (Qb)|ν1|+|ν2|tν1t2+ν2t22qν12+ν222\displaystyle\sum(-Q_{b})^{|\nu_{1}|+|\nu_{2}|}t^{\frac{||\nu_{1}^{t}||^{2}+||\nu_{2}^{t}||^{2}}{2}}q^{\frac{||\nu_{1}||^{2}+||\nu_{2}||^{2}}{2}}
×\displaystyle\times f~ν1t(q,t)f~ν2(t,q)Z~ν1t(q,t)Z~ν2t(q,t)Z~ν1(t,q)Z~ν2(t,q)\displaystyle\tilde{f}_{\nu_{1}^{t}}(q,t)\tilde{f}_{\nu_{2}}(t,q)\tilde{Z}_{\nu_{1}^{t}}(q,t)\tilde{Z}_{\nu_{2}^{t}}(q,t)\tilde{Z}_{\nu_{1}}(t,q)\tilde{Z}_{\nu_{2}}(t,q)
×\displaystyle\times i,j(1Qmqν2i+j12ti12ν2jt)i,j(1Qmq+j12ti12)i,j(1Qfqi1ν2jtjν1it)1i,j(1Qfqi1tj)1\displaystyle\frac{\prod_{i,j}(1-Q_{m}q^{-\nu_{2i}+j-\frac{1}{2}}t^{i-\frac{1}{2}-\nu_{2j}^{t}})}{\prod_{i,j}(1-Q_{m}q^{+j-\frac{1}{2}}t^{i-\frac{1}{2}})}\frac{\prod_{i,j}(1-Q_{f}q^{i-1-\nu_{2j}}t^{j-\nu_{1i}^{t}})^{-1}}{\prod_{i,j}(1-Q_{f}q^{i-1}t^{j})^{-1}}
×\displaystyle\times i,j(1Qfqjν1iti1ν2jt)1i,j(1Qfqjti1)1i,j(1QmQfqj12ν2iti12ν1jt)i,j(1QmQfqj12ti12)\displaystyle\frac{\prod_{i,j}(1-Q_{f}q^{j-\nu_{1i}}t^{i-1-\nu_{2j}^{t}})^{-1}}{\prod_{i,j}(1-Q_{f}q^{j}t^{i-1})^{-1}}\frac{\prod_{i,j}(1-Q_{m}Q_{f}q^{j-\frac{1}{2}-\nu_{2i}}t^{i-\frac{1}{2}-\nu_{1j}^{t}})}{\prod_{i,j}(1-Q_{m}Q_{f}q^{j-\frac{1}{2}}t^{i-\frac{1}{2}})}
×\displaystyle\times i,j(1QmQfqi12ν1jtj12ν2it)i,j(1QmQfqi12tj12)i,j(1QmQf2qj12ν1jti12ν1jt)i,j(1QmQf2qj12ti12)\displaystyle\frac{\prod_{i,j}(1-Q_{m}Q_{f}q^{i-\frac{1}{2}-\nu_{1j}}t^{j-\frac{1}{2}-\nu_{2i}^{t}})}{\prod_{i,j}(1-Q_{m}Q_{f}q^{i-\frac{1}{2}}t^{j-\frac{1}{2}})}\frac{\prod_{i,j}(1-Q_{m}Q_{f}^{2}q^{j-\frac{1}{2}-\nu_{1j}}t^{i-\frac{1}{2}-\nu_{1j}^{t}})}{\prod_{i,j}(1-Q_{m}Q_{f}^{2}q^{j-\frac{1}{2}}t^{i-\frac{1}{2}})}
×\displaystyle\times i,j(1+QfQmtqtν2itqi12xj)1i,j(1+Qf2Qmtqqi12tν1itxj)1\displaystyle\prod_{i,j}(1+Q_{f}Q_{m}\frac{t}{q}t^{-\nu_{2i}^{t}}q^{i-\frac{1}{2}}x_{j})^{-1}\prod_{i,j}(1+Q_{f}^{2}Q_{m}\frac{t}{q}q^{i-\frac{1}{2}}t^{-\nu_{1i}^{t}}x_{j})^{-1}
×\displaystyle\times i,j(1+qi1tν1it+12xj)i,j(1+Qfqi1tν2it+12xj)\displaystyle\prod_{i,j}(1+q^{i-1}t^{-\nu_{1i}^{t}+\frac{1}{2}}x_{j})\prod_{i,j}(1+Q_{f}q^{i-1}t^{-\nu_{2i}^{t}+\frac{1}{2}}x_{j})

Appendix B Zopenref(Qb,Qf,Qm,q,t;x)Z_{open}^{ref}(Q_{b},Q_{f},Q_{m},q,t;x): on the geometry (7): preferred direction vertical

Zopen,X2ref,ν(Q1,Q2,Q~1,Q~2,Qρ,q,t)\displaystyle Z_{open,X_{2}}^{ref,\nu}(Q_{1},Q_{2},\tilde{Q}_{1},\tilde{Q}_{2},Q_{\rho},q,t) =\displaystyle= μ1,μ2,μ1~,μ2~,ρ(Q1)|μ1|(Q2)|μ2|(Q~1)|μ~1|(Q~2)|μ~2|(Qρ)|ρ|\displaystyle\sum_{\mu_{1},\mu_{2},\tilde{\mu_{1}},\tilde{\mu_{2}},\rho}(-Q_{1})^{|\mu_{1}|}(-Q_{2})^{|\mu_{2}|}(-\tilde{Q}_{1})^{|\tilde{\mu}_{1}|}(-\tilde{Q}_{2})^{|\tilde{\mu}_{2}|}(-Q_{\rho})^{|\rho|} (95)
×\displaystyle\times Cμ~2νtμ2(t,q)Cμ~2tρμ2t(q,t)Cμ~1ρtμ1(t,q)Cμ~1tμ1t(q,t)\displaystyle C_{\tilde{\mu}_{2}\nu^{t}\mu_{2}}(t,q)C_{\tilde{\mu}_{2}^{t}\rho\mu_{2}^{t}}(q,t)C_{\tilde{\mu}_{1}\rho^{t}\mu_{1}}(t,q)C_{\tilde{\mu}_{1}^{t}\emptyset\mu_{1}^{t}}(q,t)

Using the refined topological vertex definition (3.1) and the identities in (3.4) we get the following expression for the refined open partition function

Z\displaystyle Z open,X2ref,ν(Q1,Q2,Q~1,Q~2,Qρ,q,t)=μ1,μ2(Q1)|μ1|(Q2)|μ2|t||μ1t||2+||μ2t||2+||νt||2q||μ1||2+||μ2||2||ν||2{}_{open,X_{2}}^{ref,\nu}(Q_{1},Q_{2},\tilde{Q}_{1},\tilde{Q}_{2},Q_{\rho},q,t)=\sum_{\mu_{1},\mu_{2}}(-Q_{1})^{|\mu_{1}|}(-Q_{2})^{|\mu_{2}|}\sqrt{t}^{||\mu_{1}^{t}||^{2}+||\mu_{2}^{t}||^{2}+||\nu^{t}||^{2}}\sqrt{q}^{||\mu_{1}||^{2}+||\mu_{2}||^{2}-||\nu||^{2}}
×\displaystyle\times tq|νt|Z~μ1(t,q)Z~μ1t(q,t)Z~μ2(t,q)Z~μ2t(q,t)i,j(1Q~1qρiμ1jtρjμ1it)\displaystyle\sqrt{\frac{t}{q}}^{|\nu^{t}|}\tilde{Z}_{\mu_{1}}(t,q)\tilde{Z}_{\mu_{1}^{t}}(q,t)\tilde{Z}_{\mu_{2}}(t,q)\tilde{Z}_{\mu_{2}^{t}}(q,t)\prod_{i,j}(1-\tilde{Q}_{1}q^{-\rho_{i}-\mu_{1j}}t^{-\rho_{j}-\mu_{1i}^{t}})
×\displaystyle\times i,j(1Q~2qρiμ2jtρjμ2it)i,j(1Q~ρqρiμ2jtρjμ1it)i,j(1+Q~1Qρqρiμ2jtρjμ1it)1\displaystyle\prod_{i,j}(1-\tilde{Q}_{2}q^{-\rho_{i}-\mu_{2j}}t^{-\rho_{j}-\mu_{2i}^{t}})\prod_{i,j}(1-\tilde{Q}_{\rho}q^{-\rho_{i}-\mu_{2j}}t^{-\rho_{j}-\mu_{1i}^{t}})\prod_{i,j}(1+\tilde{Q}_{1}Q_{\rho}q^{-\rho_{i}-\mu_{2j}}t^{-\rho_{j}-\mu_{1i}^{t}})^{-1}
×\displaystyle\times i,j(1+Q~2Qρqρiμ1jtρjμ1it)1i,j(1Q~1Q~2Qρqttρiμ1jtqμ2iρj)\displaystyle\prod_{i,j}(1+\tilde{Q}_{2}Q_{\rho}q^{-\rho_{i}-\mu_{1j}}t^{-\rho_{j}-\mu_{1i}^{t}})^{-1}\prod_{i,j}(1-\tilde{Q}_{1}\tilde{Q}_{2}Q_{\rho}\sqrt{\frac{q}{t}}t^{-\rho_{i}-\mu_{1j}^{t}}q^{-\mu_{2i}-\rho_{j}})
×\displaystyle\times (Q~2tq)|ν|sν(Q~21qttμ2qρ,Qρ1tμ1qρ,qtQ~1Qρqρtμ1t,qρtμ2t)\displaystyle(-\tilde{Q}_{2}\sqrt{t}{q})^{|\nu|}s_{\nu}(-\tilde{Q}_{2}^{-1}\sqrt{\frac{q}{t}}t^{\mu_{2}}q^{\rho},-Q_{\rho}^{-1}t^{\mu_{1}}q^{\rho},\sqrt{\frac{q}{t}}\tilde{Q}_{1}Q_{\rho}q^{-\rho}t^{-\mu_{1}^{t}},q^{-\rho}t^{-\mu_{2}^{t}})

Appendix C Zopenref(Qb,Qf,Qm,q,t;x)Z_{open}^{ref}(Q_{b},Q_{f},Q_{m},q,t;x): on the geometry (7): preferred direction diagonal

Zopen,X2ref,ν(Q1,Q2,Q~1,Q~2,Qρ,q,t)\displaystyle Z_{open,X_{2}}^{ref,\nu}(Q_{1},Q_{2},\tilde{Q}_{1},\tilde{Q}_{2},Q_{\rho},q,t) =\displaystyle= μ1,μ2,μ1~,μ2~,ρ(Q1)|μ1|(Q2)|μ2|(Q~1)|μ~1|(Q~2)|μ~2|(Qρ)|ρ|\displaystyle\sum_{\mu_{1},\mu_{2},\tilde{\mu_{1}},\tilde{\mu_{2}},\rho}(-Q_{1})^{|\mu_{1}|}(-Q_{2})^{|\mu_{2}|}(-\tilde{Q}_{1})^{|\tilde{\mu}_{1}|}(-\tilde{Q}_{2})^{|\tilde{\mu}_{2}|}(-Q_{\rho})^{|\rho|} (97)
×\displaystyle\times Cμ~2μ~2νt(t,q)Cμ2tμ2~tρ(q,t)Cμ1μ~1ρt(t,q)Cμ1tμ~t1(q,t)\displaystyle C_{\tilde{\mu}_{2}\tilde{\mu}_{2}\nu^{t}}(t,q)C_{\mu_{2}^{t}\tilde{\mu_{2}}^{t}\rho}(q,t)C_{\mu_{1}\tilde{\mu}_{1}\rho^{t}}(t,q)C_{\mu_{1}^{t}\tilde{\mu}^{t}_{1}\emptyset}(q,t)

Using the refined topological vertex definition (3.1) and the identities in (3.4) we get the following expression for the refined open partition function

Z\displaystyle Z open,X2ref,ν(Q1,Q2,Q~1,Q~2,Qρ,q,t)=ρ(Qρ)|μρ|Z~μρt(t,q)Z~νt(t,q)Z~μρ(q,t)t||μρ||2q||μρt||2+||νt||2{}_{open,X_{2}}^{ref,\nu}(Q_{1},Q_{2},\tilde{Q}_{1},\tilde{Q}_{2},Q_{\rho},q,t)=\sum_{\rho}(-Q_{\rho})^{|\mu_{\rho}|}\tilde{Z}_{\mu_{\rho}^{t}}(t,q)\tilde{Z}_{\nu^{t}}(t,q)\tilde{Z}_{\mu_{\rho}}(q,t)\sqrt{t}^{||\mu_{\rho}||^{2}}\sqrt{q}^{||\mu_{\rho}^{t}||^{2}+||\nu^{t}||^{2}}
×\displaystyle\times (k=1(1(Q1Q~1)k)1i,j(1(Q1Q~1)k1Q1tρiqμiρtρj)(1(Q1Q~1)k1Q~1tμiρρjqρi)(1(Q1Q~1)kqttρiqρj)(1(Q1Q~1)ktqtμiρρjqρiμjρt))\displaystyle\bigg{(}\prod_{k=1}(1-(Q_{1}\tilde{Q}_{1})^{k})^{-1}\prod_{i,j}\frac{(1-(Q_{1}\tilde{Q}_{1})^{k-1}Q_{1}t^{-\rho_{i}}q^{-\mu_{i\rho}^{t}-\rho_{j}})(1-(Q_{1}\tilde{Q}_{1})^{k-1}\tilde{Q}_{1}t^{-\mu_{i\rho}-\rho_{j}}q^{-\rho_{i}})}{(1-(Q_{1}\tilde{Q}_{1})^{k}\sqrt{\frac{q}{t}}t^{-\rho_{i}}q^{-\rho_{j}})(1-(Q_{1}\tilde{Q}_{1})^{k}\sqrt{\frac{t}{q}}t^{-\mu_{i\rho}-\rho_{j}}q^{-\rho_{i}-\mu_{j\rho}^{t}})}\bigg{)}
×\displaystyle\times (k=1(1(Q2Q~2)k)1(1(Q2Q~2)k1Q2tρiμjρqνtiρj)(1(Q2Q~2)k1Q~2tνiρjqρiμjρt)(1(Q2Q~2)kqttρiμjρqρjμiρt)(1(Q2Q~2)ktqtνiρjqρiνtj))\displaystyle\bigg{(}\prod_{k=1}(1-(Q_{2}\tilde{Q}_{2})^{k})^{-1}\frac{(1-(Q_{2}\tilde{Q}_{2})^{k-1}Q_{2}t^{-\rho_{i}-\mu_{j\rho}}q^{-\nu^{t}_{i}-\rho_{j}})(1-(Q_{2}\tilde{Q}_{2})^{k-1}\tilde{Q}_{2}t^{-\nu_{i}-\rho_{j}}q^{-\rho_{i}-\mu_{j\rho}^{t}})}{(1-(Q_{2}\tilde{Q}_{2})^{k}\sqrt{\frac{q}{t}}t^{-\rho_{i}-\mu_{j\rho}}q^{-\rho_{j}-\mu_{i\rho}^{t}})(1-(Q_{2}\tilde{Q}_{2})^{k}\sqrt{\frac{t}{q}}t^{-\nu_{i}-\rho_{j}}q^{-\rho_{i}-\nu^{t}_{j}})}\bigg{)}

Appendix D Fixed point formulae for the virtual Euler characteristics, the virtual χy\chi_{y} genus and the virtual elliptic genus

In this appendix we summarise the treatment of the virtual localisation as discussed in Fantechi_2010 . Consider a scheme XX that is equivariantly embedded into a nonsingular scheme YY globally. There is a \mathbb{C}^{*} action on the later. Let YfY^{f} denote the nonsingular fixed point locus in YY and let XfX^{f} be defined by Xf=XYfX^{f}=X\cap Y^{f}. Moreover YfY^{f} can be written as a union of irreducible components as :=iYi:=\cup_{i}Y_{i} and correspondingly Xi=XYiX_{i}=X\cap Y_{i}. We will denote by [Xi]vir[X_{i}]^{vir} the virtual fundamental class, by 𝒪Xivir\mathcal{O}_{X_{i}}^{vir} the virtual structure sheaf and by NivirN_{i}^{vir} the virtual normal bundle of XiX_{i}. For the vector bundle VV defined on XX777Strictly speaking VV is an element of the Grothendieck group of the vector bundles on X. the virtual pushforward pvir()p_{*}^{vir}() is related to the pushforward p()p_{*}() by

pvir(V)=p(ch(V)td(TvirX)[X]vir)\displaystyle p_{*}^{vir}(V)=p_{*}(ch(V)td(T^{vir}_{X})\cap[X]^{vir}) (99)

Consider an equivariant lift of VV denoted by V~\tilde{V}. We consider the restriction of V~\tilde{V} to XiX_{i} denoted by V~i\tilde{V}_{i} and a projection pi:Xiptp_{i}:X_{i}\to pt. Here and below we denote by ch the equivariant Chern character, by td the equivariant Todd genus, by i0(1)iΛiB\sum_{i\geq 0}(-1)^{i}\Lambda^{i}B the action of Λ1\Lambda_{-1} on the vector bundle B and by Eu the equivariant Euler class.
The virtual Riemann-Roch theorem gives

χvir(X,V)=[X]virch(V).td(TXvir)=p(ch(V~).td(TvirX)[X]vir)|ϵ=0\displaystyle\chi^{vir}(X,V)=\int_{[X]^{vir}}\mbox{ch}(V).\mbox{td}(T_{X}^{vir})=p_{*}(\mbox{ch}(\tilde{V}).\mbox{td}(T^{vir}_{X})\cap[X]^{vir})|_{\epsilon=0} (100)

where the parameter ϵ\epsilon is related to the equivariant lift V~\tilde{V}. Then by applying the localisation formula we get

p(ch(V~).td(TX)vir[X]vir)\displaystyle p_{*}(\mbox{ch}(\tilde{V}).\mbox{td}(T_{X})^{vir}\cap[X]^{vir}) =\displaystyle= ipi(ch(V~i)td(TXvir|Xi)/Eu(Nivir)[X]viri)\displaystyle\sum_{i}p_{i*}\big{(}\mbox{ch}(\tilde{V}_{i})\mbox{td}(T_{X}^{vir}|_{X_{i}})/\mbox{Eu}(N_{i}^{vir})\cap[X]^{vir}_{i}\big{)} (101)
=\displaystyle= ipi(td(TXivir)ch(Vi~/ch(Λ1(Nivir)))[Xi]vir)\displaystyle\sum_{i}p_{i*}\big{(}\mbox{td}(T_{X_{i}}^{vir})\mbox{ch}(\tilde{V_{i}}/\mbox{ch}(\Lambda_{-1}(N_{i}^{vir}))^{\vee})\cap[X_{i}]^{vir}\big{)}

Using the following identities

td(TXvir|Xi)\displaystyle\mbox{td}(T_{X}^{vir}|_{X_{i}}) =\displaystyle= td(TXivir)td(Nivir)\displaystyle\mbox{td}(T_{X_{i}}^{vir})\mbox{td}(N_{i}^{vir})
td(Nivir)\displaystyle\mbox{td}(N_{i}^{vir}) =\displaystyle= Eu(Nivir)/ch(Λ1(Nivir))\displaystyle\mbox{Eu}(N_{i}^{vir})/\mbox{ch}(\Lambda_{-1}(N_{i}^{vir})^{\vee}) (102)

and the eq.(99) we get

p(ch(V~).td(TX)vir[X]vir)\displaystyle p_{*}(\mbox{ch}(\tilde{V}).\mbox{td}(T_{X})^{vir}\cap[X]^{vir}) =\displaystyle= ipivir(V~i/Λ1(Nivir))\displaystyle\sum_{i}p_{i*}^{vir}(\tilde{V}_{i}/\Lambda_{-1}(N_{i}^{vir})^{\vee}) (103)

By definition

χvir(X,V~,ϵ):=ipivir(V~i/Λ1(Nivir))\displaystyle\chi^{vir}(X,\tilde{V},\epsilon):=\sum_{i}p_{i*}^{vir}(\tilde{V}_{i}/\Lambda_{-1}(N_{i}^{vir})^{\vee}) (104)

Moreover the eq.(103) implies that χvir(X,V,ϵ)Q[[ϵ]]\chi^{vir}(X,V,\epsilon)\in Q[[\epsilon]] and χvir(X,V)=χvir(X,V~,0)\chi^{vir}(X,V)=\chi^{vir}(X,\tilde{V},0).
Similar manipulations lead to the fixed point formulas for the chichi-y genus and ellipticelliptic genus.

χyvir(X,V)\displaystyle\chi_{-y}^{vir}(X,V) =\displaystyle= (iχyvir(Xi,V~iΛy(Nivir)/Λ1(Nivir)))|ϵ=0\displaystyle\big{(}\sum_{i}\chi_{-y}^{vir}(X_{i},\tilde{V}_{i}\otimes\Lambda_{-y}(N_{i}^{vir})^{\vee}/\Lambda_{-1}(N_{i}^{vir})^{\vee})\big{)}|_{\epsilon=0}

If we define ni=rank(Nivir)n_{i}=rank(N_{i}^{vir}) then

Ellvir(X;z,τ)=(iyni/2Ellvir(Xi,(Nivir)Λy(Nivir)/Λ1(Nivir),z,τ))|ϵ=0\displaystyle Ell^{vir}(X;z,\tau)=\big{(}\sum_{i}y^{-n_{i}/2}Ell^{vir}(X_{i},\mathcal{E}(N_{i}^{vir})\Lambda_{-y}(N_{i}^{vir})/\Lambda_{-1}(N_{i}^{vir})^{\vee},z,\tau)\big{)}|_{\epsilon=0} (106)

References

  • [1] Mina Aganagic, Albrecht Klemm, Marcos Marino, and Cumrun Vafa. The Topological vertex. Commun. Math. Phys., 254:425–478, 2005.
  • [2] Ignatios Antoniadis, E. Gava, K.S. Narain, and T.R. Taylor. Topological amplitudes in string theory. Nucl. Phys. B, 413:162–184, 1994.
  • [3] Brice Bastian, Stefan Hohenegger, Amer Iqbal, and Soo-Jong Rey. Five-Dimensional Gauge Theories from Shifted Web Diagrams. Phys. Rev. D, 99(4):046012, 2019.
  • [4] Giulio Bonelli, Nadir Fasola, and Alessandro Tanzini. Flags of sheaves, quivers and symmetric polynomials. 11 2019.
  • [5] U. Bruzzo, W.-y. Chuang, D.-E. Diaconescu, M. Jardim, G. Pan, and Y. Zhang. D-branes, surface operators, and ADHM quiver representations. Adv. Theor. Math. Phys., 15(3):849–911, 2011.
  • [6] Wu-yen Chuang, Duiliu-Emanuel Diaconescu, Ron Donagi, and Tony Pantev. Parabolic refined invariants and Macdonald polynomials. Commun. Math. Phys., 335(3):1323–1379, 2015.
  • [7] Wu-yen Chuang, Duiliu-Emanuel Diaconescu, and Guang Pan. Wallcrossing and Cohomology of The Moduli Space of Hitchin Pairs. Commun. Num. Theor. Phys., 5:1–56, 2011.
  • [8] W.Y. Chuang, D.E. Diaconescu, and G. Pan. BPS states and the P=W conjecture. Lond. Math. Soc. Lect. Note Ser., 411:132–150, 2014.
  • [9] Barbara Fantechi and Lothar Gottsche. Riemann?roch theorems and elliptic genus for virtually smooth schemes. Geometry and Topology, 14(1):83?115, Jan 2010.
  • [10] V. Gritsenko. Complex vector bundles and Jacobi forms. 1 1999.
  • [11] Babak Haghighat, Amer Iqbal, Can Kozçaz, Guglielmo Lockhart, and Cumrun Vafa. M-Strings. Commun. Math. Phys., 334(2):779–842, 2015.
  • [12] J. Haglund. A combinatorial model for the macdonald polynomials. Proceedings of the National Academy of Sciences, 101(46):16127?16131, Nov 2004.
  • [13] Mark Haiman. Hilbert schemes, polygraphs, and the macdonald positivity conjecture, 2000.
  • [14] Timothy J. Hollowood. Five-dimensional gauge theories and quantum mechanical matrix models. JHEP, 03:039, 2003.
  • [15] Timothy J. Hollowood, Amer Iqbal, and Cumrun Vafa. Matrix models, geometric engineering and elliptic genera. JHEP, 03:069, 2008.
  • [16] Amer Iqbal. Unpublished notes.
  • [17] Amer Iqbal and Amir-Kian Kashani-Poor. The Vertex on a strip. Adv. Theor. Math. Phys., 10(3):317–343, 2006.
  • [18] Amer Iqbal, Ahsan Z. Khan, Babar A. Qureshi, Khurram Shabbir, and Muhammad A. Shehper. Topological field theory amplitudes for AN-1 fibration. JHEP, 12:017, 2015.
  • [19] Amer Iqbal and Can Kozcaz. Refined topological strings on local 2{\mathrm{\mathbb{P}}}^{2}. JHEP, 03:069, 2017.
  • [20] Amer Iqbal, Can Kozcaz, and Khurram Shabbir. Refined Topological Vertex, Cylindric Partitions and the U(1) Adjoint Theory. Nucl. Phys. B, 838:422–457, 2010.
  • [21] Amer Iqbal, Can Kozcaz, and Cumrun Vafa. The Refined topological vertex. JHEP, 10:069, 2009.
  • [22] Amer Iqbal and Cumrun Vafa. BPS Degeneracies and Superconformal Index in Diverse Dimensions. Phys. Rev. D, 90(10):105031, 2014.
  • [23] Can Kozçaz, Shamil Shakirov, Cumrun Vafa, and Wenbin Yan. Refined Topological Branes. 5 2018.
  • [24] Jun Li, Kefeng Liu, and Jian Zhou. Topological string partition functions as equivariant indices. 12 2004.
  • [25] Ian Grant MacDonald and A V Zelevinsky. Symmetric functions and Hall polynomials; 2nd ed. Oxford classic texts in the physical sciences. Oxford University Press, Oxford, 1998.
  • [26] Andrew Neitzke and Cumrun Vafa. Topological strings and their physical applications. 10 2004.
  • [27] Yuji Sugimoto. The Enhancement of Supersymmetry in M-strings. Int. J. Mod. Phys. A, 31(16):1650088, 2016.
  • [28] Masato Taki. Flop Invariance of Refined Topological Vertex and Link Homologies. 5 2008.
  • [29] Ahmad Zein Assi. Topological Amplitudes and the String Effective Action. PhD thesis, Ecole Polytechnique, CPHT, 2013.