Based on verse 22, it appears there was immediately a double-reaction: some were amazed and part of their amazement at his "gracious" speech gets expressed in the line "Isn't this Joseph's son?" But that question seems to cut two ways, and Jesus' subsequent words indicate his awareness of this. The question "Isn't this Joseph's son" CAN be a source of genuine wonder and appreciation—look how far our local boy has come! But it's not difficult to see that the same question could be asked with a real edge to it, with a sneer, with derision. "Joseph's kid? Good grief. He was a nobody back in the day and he's a nobody from a no-account family now. Forget him!"
Jesus then goes on to suggest that maybe those very detractors in the crowd that day would be asking him shortly for an authenticating sign. Although we have not as of yet been told directly by Luke of any particular work Jesus didin Capernaum, apparently he's been there and done some amazing things. But Jesus is no trained dog or dancing bear and he makes clear he's not going to do any such thing in Nazareth. Worse, he inflames people still more by saying that with the attitudes some were harboring in their hearts at that very moment, the Nazareth populace was not worthy of a divine working. Instead, as in the feckless, sub-spiritual days of Elijah and Elisha, God would work his wonders elsewhere, outside Israel.