Exhibit 2.h.3

GLADSTONE CAPITAL CORPORATION

AMENDMENT NO. 1 TO

EQUITY DISTRIBUTION AGREEMENT

May 22, 2017

Cantor Fitzgerald & Co.

499 Park Avenue

New York, NY 10022

Ladies and Gentlemen:

Reference is made to the Equity Distribution Agreement, dated February 27, 2015, including the Schedules thereto (the “Equity Distribution Agreement”), by and among Gladstone Capital Corporation, a Maryland corporation (the “Company”), Gladstone Management Corporation, a Delaware corporation registered as an investment adviser (the “Adviser”), Gladstone Administration, LLC, a Delaware limited liability company (the “Administrator”), and Cantor Fitzgerald & Co. (the “Agent”), pursuant to which the Company agreed to sell through the Agent, as sales agent, shares of common stock, par value $0.001 per share, of the Company. All capitalized terms used in this Amendment No. 1 to Equity Distribution Agreement among the Company, the Adviser, the Administrator, and the Agent (this “Amendment”) and not otherwise defined herein shall have the respective meanings assigned to such terms in the Equity Distribution Agreement. The Company, the Adviser, the Administrator, and the Agent agree as follows:

A. Amendments to Equity Distribution Agreement. On or after the date hereof (the “Amendment Date”), the Equity Distribution Agreement is amended as follows:

1. All references to “KeyBanc” or “the KeyBanc Agreement” in the Equity Distribution Agreement shall be deleted and any sentences containing such references shall be amended to refer to “the Agent” or “this Agreement” in the singular.

2. The second paragraph of Section 1 is hereby deleted and replaced with the following text:

“The aggregate offering price of Shares that may be sold collectively pursuant to this Agreement and a prior equity distribution agreement, dated February 27, 2015, by and among the Company, the Adviser, the Administrator and KeyBanc Capital Markets Inc., shall not exceed $50,000,000.”

3. The first sentence of the third paragraph of Section 1 is amended by adding “, as amended on October 13, 2015” immediately after “October 1, 2006”.

4. The first sentence of the fourth paragraph of Section 1 is amended by deleting the number “333-185191” and replacing it with “333-208637”.

5. Section 2(b) is amended by deleting the date “December 31, 2014” and replacing it with “March 31, 2017”.

6. The first sentence of Section 2(d) is deleted in its entirety.

7. Section 2(hh) is deleted in its entirety and replaced with the following:

“(hh) Except as disclosed in the Registration Statement and the Prospectus, as required to comply with the 1940 Act, as required to comply with the Fifth Amended and Restated Credit Agreement, dated as of May 1, 2015, by and among Gladstone Business Loan, LLC, as borrower, the Adviser, as servicer, the lenders and managing agents named therein, and Keybank National Association, as administrative agent, as the same may be as amended or restated from time to time, and for such prohibitions that would not reasonably be expected to result in a Company Material Adverse Effect, no Subsidiary, other than Gladstone Business Loan LLC, is currently prohibited, directly or indirectly, from paying any dividends to the Company, from making any other distribution on such Subsidiary’s capital stock, from repaying to the Company any loans or advances to such subsidiary from the Company or from transferring any of such Subsidiary’s property or assets to the Company or any other Subsidiary.”

8. The last sentence of Section 2(mm) is deleted in its entirety and replaced with the following “Notwithstanding the foregoing, as of the date of this Agreement, the Company has not filed this Agreement as an exhibit to the Registration Statement, although all such exhibits will be timely filed by post-effective amendment pursuant to Rule 462(d) under the 1933 Act.”

9. Section 4(f) is amended by adding “, and in each case, in accordance with applicable rules and regulations” immediately before “(each such day, a “Settlement Date”)”.

10. Section 4(h) of the Equity Distribution Agreement is hereby deleted in its entirety and replaced with the following: “[Reserved]”.

11. Section 6(p) is amended by deleting “K&L Gates LLP” and replacing it with “Stradley Ronon Stevens and Young, LLP”.

12. Section 6(s) is amended by deleting “Troutman Sanders LLP” and replacing it with “Cooley LLP”.

13. Section 6(x) is deleted in entirety and replaced with the following:

“(x) Absent the prior written consent of the Agent, during the term of this Agreement, the Company will not enter into another agreement for an At the Market Offering program with any other party, other than the Agent.”

14. Section 8(a) is amended by deleting the number “(i)” and the following language: “or (ii) the Agent’s willful misfeasance, bad faith or gross negligence in the performance of the Agent’s duties or by reckless disregard of its obligations and duties under the Agreement”.

-2-

15. Section 11 is amended by: (i) adding “and (212) 829-4798” immediately after “(212) 308-3730”; (ii) deleting the words “Jeffrey Lumby” and replacing them with “Capital Markets/ Jeffrey Lumby and General Counsel, respectively”; and (iii) deleting the words “Troutman Sanders LLP, 1001 Haxall Point, Richmond, Virginia 23219, fax no. (804) 698-5185, Attention: Michael T. Damgard” and replacing them with “Cooley LLP, 1114 Avenue of the Americas, New York, NY 10036, fax. no (212) 479-6275, Attention: Daniel I. Goldberg”.

16. Section 15 is deleted and replaced with the following:

“Applicable Law; Consent to Jurisdiction. This Agreement will be governed by and construed in accordance with the laws of the State of New York applicable to contracts made and to be performed within the State of New York. Each party hereby irrevocably submits to the non-exclusive jurisdiction of the state and federal courts sitting in the City of New York, Borough of Manhattan, for the adjudication of any dispute hereunder or in connection with any transaction contemplated hereby, and hereby irrevocably waives, and agrees not to assert in any suit, action or proceeding, any claim that it is not personally subject to the jurisdiction of any such court, that such suit, action or proceeding is brought in an inconvenient forum or that the venue of such suit, action or proceeding is improper. Each party hereby irrevocably waives personal service of process and consents to process being served in any such suit, action or proceeding by mailing a copy thereof (certified or registered mail, return receipt requested) to such party at the address in effect for notices to it under this Agreement and agrees that such service shall constitute good and sufficient service of process and notice thereof. Nothing contained herein shall be deemed to limit in any way any right to serve process in any manner permitted by law.”

17. Section 21 of the Equity Distribution Agreement is hereby deleted in its entirety.

18. The first sentence of the form of Company Officer Certificate and the form of Adviser Certificate, respectively, attached as Exhibit 6(n) are amended to (i) add the words “February 27, 2015, as amended on May 22, 2017” immediately after “dated”, and (ii) add “Gladstone Administration, LLC” immediately after “Gladstone Management Corporation”.

B. Prospectus Supplement. The Company shall prepare and file in accordance with Rule 497 of the Securities Act, a Prospectus Supplement reflecting this Amendment.

C. Notwithstanding anything to the contrary contained herein, this Amendment shall not have any effect on the terms of the Equity Distribution Agreement prior to the Amendment Date, and the rights and obligations of the parties thereunder, including, without limitation, the representations, warranties and agreements (including the indemnification and contribution provisions), as well as the definitions of “Registration Statement,” “Base Prospectus,” “Prospectus Supplement” and “Prospectus,” contained in the Equity Distribution Agreement.

-3-

D. No Other Amendments. Except as set forth in Part A above, all the terms and provisions of the Equity Distribution Agreement shall continue in full force and effect.

E. Counterparts. This Amendment may be executed in two or more counterparts, each of which shall be deemed an original, but all of which together shall constitute one and the same instrument. Delivery of an executed Amendment by one party to the other may be made by facsimile or email transmission.

F. Governing Law. This Amendment shall be governed by, and construed in accordance with, the internal laws of the State of New York without regard to the principles of conflicts of laws.

[Remainder of page intentionally left blank.]

-4-

If the foregoing correctly sets forth the understanding between us, please so indicate in the space provided below for that purpose.