We attempt to discuss a new circle problem. Let
$\unicode[STIX]{x1D701}(s)$ denote the Riemann zeta-function
$\sum _{n=1}^{\infty }n^{-s}$ (
$\text{Re}\,s>1$) and
$L(s,\unicode[STIX]{x1D712}_{4})$ the Dirichlet
$L$-function
$\sum _{n=1}^{\infty }\unicode[STIX]{x1D712}_{4}(n)n^{-s}$ (
$\text{Re}\,s>1$) with the primitive Dirichlet character mod 4. We shall define an arithmetical function
$R_{(1,1)}(n)$ by the coefficient of the Dirichlet series
$\unicode[STIX]{x1D701}^{\prime }(s)L^{\prime }(s,\unicode[STIX]{x1D712}_{4})=\sum _{n=1}^{\infty }R_{(1,1)}(n)n^{-s}$
$(\text{Re}\,s>1)$. This is an analogue of
$r(n)/4=\sum _{d|n}\unicode[STIX]{x1D712}_{4}(d)$. In the circle problem, there are many researches of estimations and related topics on the error term in the asymptotic formula for
$\sum _{n\leq x}r(n)$. As a new problem, we deduce a ‘truncated Voronoï formula’ for the error term in the asymptotic formula for
$\sum _{n\leq x}R_{(1,1)}(n)$. As a direct application, we show the mean square for the error term in our new problem.