It was while Mortimor, Marcus and Wyatt Jeffries were in Greensville County, Virginia, that harsh laws were passed against "people of color", as they were classified by law because they were part Indian. This put them legally in the same group as free persons of African or part African descent. These events no doubt account for the fact that in the 1840 and 1850 Smith Township census, Jeffries, Pompey and Jones families were listed in a column headed "Free Colored Persons".

In some of these families, there were problems with education because of the assumed African extraction. This sentiment was so strong among the Jeffries neighbors that the right of suffrage was refused them until 1860, when this family voted for Lincoln against the most urgent protests and demonstrations of their neighbors. To prevent a repetition of their again exercising the right of suffrage, the citizens of the township elected Wells Smith, a Republican, as trustee, who declared that if elected, he would prevent them from exercising their rights by refusing to take their ballots.

This question of suffrage in connection with the strenuous times of the Civil War, created a political furor among all parties. The refusal of Mortimor Jeffries' ballot by Trustee Smith, was the straw that broke the camel's back, and he immediately resorted to the intercession of the courts. The case, on change of venue, was taken to Noble County, where it was bitterly fought by the best legal talent obtainable, but Mortimor lost out.

During the trial, one witness assumed to be an expert in distinguishing traces of African blood by a critical examination of the hair. Mr. Jeffries' attorney presented to this witness a lock of hair clipped from the judge's head, which the witness, after a careful examination pronounced to be African hair.

Mr. Jeffries did not lie down supinely, but being more determined to secure his rights, carried his case to the Supreme Court and was granted suffrage for himself and brothers, which they afterward exercised undisputed under the scornful eyes of some of their neighbors. Mortimor Jeffries fought his legal battle for the rights of himself and brothers, as descendants of Indian and French.