The majority of infections with SARS-CoV-2 are asymptomatic or mild without the necessity of hospitalization. It is of importance to reveal if these patients develop an antibody response against SARS-CoV-2 and to define which antibodies confer virus neutralization. We conducted a comprehensive serological survey of 49 patients with a mild course of disease and quantified neutralizing antibody responses against a clinical SARS-CoV-2 isolate employing human cells as targets. Four patients (8%), even though symptomatic, did not develop antibodies against SARS-CoV-2, and two other patients (4%) were positive in only one of the six serological assays employed. For the remaining 88%, antibody response against the S protein correlated with serum neutralization whereas antibodies against the nucleocapsid were poor predictors of virus neutralization. None of the sera enhanced infection of human cells with SARS-CoV-2 at any dilution, arguing against antibody-dependent enhancement of infection in our system. Regarding neutralization, only six patients (12%) could be classified as high neutralizers. Furthermore, sera from several individuals with fairly high antibody levels had only poor neutralizing activity. In addition, employing a novel serological Western blot system to characterize antibody responses against seasonal coronaviruses, we found that antibodies against the seasonal coronavirus 229E might contribute to SARS-CoV-2 neutralization. Altogether, we show that there is a wide breadth of antibody responses against SARS-CoV-2 in patients that differentially correlate with virus neutralization. This highlights the difficulty to define reliable surrogate markers for immunity against SARS-CoV-2.IMPORTANCE There is strong interest in the nature of the neutralizing antibody response against SARS-CoV-2 in infected individuals. For vaccine development, it is especially important which antibodies confer protection against SARS-CoV-2, if there is a phenomenon called antibody-dependent enhancement (ADE) of infection, and if there is cross-protection by antibodies directed against seasonal coronaviruses. We addressed these questions and found in accordance with other studies that neutralization is mediated mainly by antibodies directed against the spike protein of SARS-CoV-2 in general and the receptor binding site in particular. In our test system, utilizing human cells for infection experiments, we did not detect ADE. However, using a novel diagnostic test we found that antibodies against the coronavirus 229E might be involved in cross-protection to SARS-CoV-2.