In the present paper, we provide a theoretical approach to the analysis of nanopore morphological stability under diffusion taking into account surface elasticity. It is assumed that the pore shape may change during stress relaxation. The surface atomic flux is caused by the nonuniform distribution of chemical potential, which depends on bulk and surface stresses as well as pore surface curvature. The change in surface relief is described by the linearized evolution equation, which is derived taking into account the solution of the boundary value problem of plane elasticity for determining the elastic and surface energy. For this purpose, we use a first-order approximation of the boundary perturbation method. The solution of the linearized evolution equation allows us to analyze the influence of physical and geometrical parameters of the problem on pore surface morphological stability.