States Permitting Common Law Marriage as of 2006:

Alabama , Colorado, District of Columbia, Iowa, Kansas, Montana, Oklahoma, Pennsylvania, Rhode Island, South Carolina, Texas, and Utah .

States Permitting Certain (older) Common Law Marriages

  Georgia (if the elements were satisfied before January 1, 1997) 
  Idaho (if the elements were satisfied before January 1, 1996) 
  New Hampshire (for inheritance only) 
  Ohio (if the elements were satisfied before October 10, 1991) 

§ 7703. Determination of marital status.

(a) General rule. For purposes of ... this title ...--

(1) the determination of whether an individual is married shall be made as of the close of his taxable year; except that if his spouse dies during his taxable year such determination shall be made as of the time of such death; and

(2) an individual legally separated from his spouse under a decree of divorce or of separate maintenance shall not be considered as married.

§ 6013. Joint returns of income tax by husband and wife.

(a) Joint returns. A husband and wife may make a single return jointly of income taxes under subtitle A, even though one of the spouses has neither gross income nor deductions, except as provided below:

(1) no joint return shall be made if either the husband or wife at any time during the taxable year is a nonresident alien;

The rest of the section has not been reproduced here as it is not relevant to the question.

BNA, a nationally recognized tax service states the following at ¶3310.04.H.

Determining Marital Status

1. In general

The determination of an individual's marital status is made on the last day of his or her taxable year. ²²⁷ Thus, if taxpayers marry on or before December 31, they are considered married for the entire taxable year. ²²⁸ However, if an individual's spouse dies during the taxable year, marital status is determined on the date of the spouse's death. ²²⁹

Generally, individuals are married for federal income tax purposes if they are considered married under state law. ²³⁰ However, for purposes of interpreting any federal statute, regulation, or ruling, a marriage requires a legal union between one man and one woman as husband and wife, and a spouse is a person of the opposite gender.²³¹

Common law marriages are recognized for federal income tax purposes if they are recognized by the state in which the taxpayers reside. ²³² If the taxpayers later move to a state which does not recognize common law marriages, they are still considered married for federal income tax purposes. ²³³ However, if the taxpayers begin and maintain such relationship in a state which does not recognize common law marriages, they will not be considered married. ²³⁴

Taxpayers are not married for federal income tax purposes when they are legally separated under a decree of divorce or of separate maintenance. ²³⁵ If a taxpayer introduces into evidence a marriage certificate, the burden of proving that the taxpayer is legally separated shifts to the IRS. ²³⁶ A legal separation requires a final divorce decree or a decree of separate maintenance. ²³⁷

Taxpayers who obtain a divorce solely for purposes of filing individual income tax returns as single taxpayers are considered married. ²³⁸ Thus, taxpayers who obtain a divorce at the end of the year and remarry in a subsequent year are considered married for federal income tax purposes. ²³⁹ Otherwise, the IRS generally does not question the validity of any divorce decree until a court of competent jurisdiction declares the divorce to be invalid. ²⁴⁰ Taxpayers who file joint tax returns while married and later have their marriage annulled must file amended returns as single taxpayers. ²⁴¹

Footnotes:

²²⁷ §7703(a)(1).

²²⁸ Id.

²²⁹ Id.

²³⁰ E.g., Rev. Rul. 58-66, 1958-1 C.B. 60.

²³¹ 1 U.S.C. §7 (1996).

²³² Rev. Rul. 58-66, 1958-1 C.B. 60.

²³³ Id.

²³⁴ E.g., Peacock v. Comr., T.C. Memo 1978-30.

²³⁵ §7703(a)(2). See §6013(d)(2); Regs. §1.7703-1(b)(4).

²³⁶ Moretti v. Comr., 77 F.3d 637 (2d Cir. 1996).

²³⁷ Id.

²³⁸ Rev. Rul. 76-255, 1976-2 C.B. 40.

²³⁹ Id.

²⁴⁰ Rev. Rul. 67-442, 1967-2 C.B. 65.

²⁴¹ Rev. Rul. 76-255, 1976-2 C.B. 40.

Another resource discusses marriage as follows:

“Whether a marriage is recognized for tax purposes depends on state law. Thus, if taxpayers are married in compliance with the laws of the state in which they are married, then the marriage is recognized for tax purposes, even if they later reside in another state. …  If local law recognizes common law marriage, taxpayers with a common law marriage are considered married for tax purposes, even if they later move to a state that does not recognize common law marriage. Rev. Rul. 58-66. 

REV. RUL. 58-66

   The marital status of individuals as determined under state law is recognized in the administration of the Federal income tax laws. Therefore, if applicable state law recognizes common-law marriages, the status of individuals living in such relationship that the state would treat them as husband and wife is, for Federal income tax purposes, that of husband and wife.

   The foregoing position of the Internal Revenue Service with respect to a common-law marriage is equally applicable in the case of taxpayers who enter into a common-law marriage in a state which recognizes such relationship and who later move into a state in which a ceremony is required to initiate the marital relationship. Accordingly, a taxpayer who enters into a common-law marriage in a state which recognizes such marriages is entitled, under the provisions of section 151(b) of the Internal Revenue Code of 1954, to an exemption of $600 for his common-law wife in making a separate income Tax return, provided that, for the calendar year in which the taxable year of the taxpayer begins, she has no gross income and is not the dependent of another taxpayer. Also, for the purpose of filing a joint income tax return under section 6013(a) of the Code, a common-law wife in a state which recognizes such marriages will be considered to be the taxpayer's spouse.