Veliparib and Topotecan Hydrochloride in Treating Patients With Solid Tumors, Relapsed or Refractory Ovarian Cancer, or Primary Peritoneal Cancer

Inclusion Criteria:

PHASE I: Adult patients with histologically confirmed solid tumor malignancy that is metastatic or unresectable and for which standard curative measures or other therapy definitely capable of extending life expectancy does not exist
PHASE II: All patients enrolled in the Phase II portion of this trial must have a history of biopsy-proven ovarian, fallopian tube or primary peritoneal cancer
Patients must have received < 3 lines of prior therapy and have relapsed less than a year from their last platinum regimen; regimens that are used twice (for example carboplatin and paclitaxel) can be counted as one; if a regimen is changed during the course of treatment due to side effect profile or allergy, the course of therapy is counted as one regimen; (for example, if docetaxel is substituted for paclitaxel due to a reaction during the initial course of adjuvant therapy, this is considered one regimen)
Patients must have measurable disease with at least one lesion whose longest diameter can be accurately measured as >= 2.0 cm with conventional techniques or as >= 1.0 cm with spiral computed topography (CT); if spiral CT is used, it must be used for both pre- and post- treatment tumor assessments
Absolute neutrophil count >= 1500/mcL
Hemoglobin >= 9.0 g/dL
Platelets >= 100,000/mcL
Total bilirubin =< 1.5 x the upper limit of normal (ULN)
Serum glutamic pyruvic transaminase (SGPT) (alanine aminotransferase [ALT]) or serum glutamic oxaloacetic transaminase (SGOT) (aspartate aminotransferase [AST]) =< 2.5 x ULN in the absence of hepatic metastasis; SGPT (ALT) =< 3 x ULN or SGOT (AST) =< 5 x ULN in the presences of hepatic metastasis
Creatinine =< 1.5 x ULN
International normalized ratio (INR) =< 1.4 unless receiving therapeutic doses of coumadin
Partial thromboplastin time (PTT) =< 48 seconds (1.25 x ULN)
Eastern Cooperative Oncology Group (ECOG) performance status (PS) 0, 1, or 2
Ability to provide informed consent
Willingness to return to enrolling institution for follow up
Life expectancy >= 12 weeks

Willingness to provide the biologic specimens is required by the protocol; this is part of the mandatory correlative research component; these specimens include:

PHASE I: peripheral blood for plasma pharmacokinetic analysis and peripheral blood mononuclear cell (PBMC) polymer assessment from 0-24 h after drug administration on days 1 and 2 of cycle 1 as well as day 2 of cycle 2; urine for assessment of ABT-888 renal clearance for 24 h after administration of drugs on days 1 and 2 of cycle 1 as well as day 2 of cycle 2; and a pretreatment peripheral blood sample for possible sequencing of the BRCA1, BRCA2 loci as well as possible pharmacogenomic analysis
Negative urine or serum pregnancy test done =< 7 days prior to registration for females of child bearing potential only
Able to swallow and absorb the medication

Exclusion Criteria:

Known standard therapy for the patient's disease that is potentially curative or definitely capable of extending life expectancy
Prior treatment with a PARP inhibitor or topotecan
Uncontrolled intercurrent illness including, but not limited to, ongoing or active infection, symptomatic congestive heart failure, unstable angina pectoris, cardiac arrhythmia, or psychiatric illness/social situations that would limit compliance with study requirements

Any of the following prior therapies:

Chemotherapy =< 4 weeks prior to registration
Mitomycin C/nitrosoureas =< 6 weeks prior to registration
Immunotherapy =< 4 weeks prior to registration
Biologic therapy =< 4 weeks prior to registration
Radiation therapy =< 4 weeks prior to registration
Radiation to > 25% of bone marrow
Investigational therapy or any ancillary therapy considered investigational (utilized for a non-Food and Drug Administration [FDA]-approved indication and in the context of a research investigation) =< 4 weeks prior to registration; subjects with prostate cancer will be permitted to continue hormone therapy
Failure to fully recover from acute, reversible effects of prior chemotherapy regardless of interval since last treatment
New York Heart Association classification III or IV
Known central nervous system (CNS) metastases or seizure disorder; patients with known brain metastases that have been successfully treated and stable for >= 6 months without requirement for corticosteroids and without seizure activity will be eligible

Any of the following:

Pregnant women
Nursing women
Men or women of childbearing potential who are unwilling to employ adequate contraception
Co-morbid systemic illnesses or other severe concurrent disease which, in the judgment of the investigator, would make the patient inappropriate for entry into this study or interfere significantly with the proper assessment of safety and toxicity of the prescribed regimens
Immunocompromised patients (other than that related to the use of corticosteroids) including patients known to be human immunodeficiency virus (HIV) positive
Receiving any other investigational agent which would be considered as a treatment for the primary neoplasm

Other active malignancy, except non-melanotic skin cancer or carcinoma-in-situ of the cervix

Note: If there is a history of prior malignancy, they must not be receiving other specific treatment (other than hormonal therapy) for their cancer
History of myocardial infarction =< 6 months, or congestive heart failure requiring use of ongoing maintenance therapy for life-threatening ventricular arrhythmias

More than 2 prior chemotherapy regimens for the current malignancy; full dose chemotherapy used in conjunction with concurrent radiation therapy will be included as prior therapy

Note: Prior hormonal therapy (e.g. leuprolide, aromatase inhibitors, tamoxifen) and immunotherapy will be allowed and not included as a prior chemotherapy; if the chemotherapy regimen is altered during the course due to issues with tolerability or safety, the regimen will be counted as one; using the same regimen at recurrence is counted as one regimen; the addition of bevacizumab to a prior regimen is considered one regimen