I am trying to setup port forwarding for ssh through 2 NATs

The first Router translates my internet IP to my outer network (10.1.7.0). In the outer network there's a second Router that does NAT to my inner network (192.168.1.0).

The target server is connected to both, the outer network and the inner network.

I cannot change the port forwarding options for outer router. It is currently configured to forward the SSH and HTTP port to the router for the inner network.

               Internet
                    +
                    |
                    v
            +-----------------+                      +------------------+
            |  Outer Router   |                      |   Inner Router   |
            |-----------------|                      |------------------|
            |                 |       SSH HTTP       |                  |
       +----+                 +--------------------->|                  |
       |    |                 |                      |                  |
       |    |                 |                      |                  |
       |    +-------+---------+                      +------+---------+-+
       |            |                                       |         |
       |            |                                       |         |
       |            |                                       |         |
       |            |           +------------------+        | SSH     |
       |            |           |   Server         |        |         |
       |            |           |------------------|        |         |
       |            +----------->                  |<-------+         |
       |                        |                  |                  |HTTP (testing)
       |                        +------------------+                  |
       |                                                              |
+------v------------------+                                           |
|  Outer Workstation      |     +-------------------+                 |
|-------------------------|     |  Inner Workstation|                 |
|                         |     |-------------------|                 |
|                         |     |                   |<----------------+
+-------------------------+     |                   |
                                +-------------------+

When connecting from a outer workstation to the address of the inner router, then both SSH and HTTP work fine.

When connecting from the internet to my public ip with HTTP, the connection works fine as well.

However SSH just times out. Most likely because the reply is not routed back properly.

I suspect its either because of the SSH itself, or because the server is connected to both, the inner and outer network.

Any ideas how I could resolve this issue?

The routes on the server are currently:

ip route show
default via 10.1.7.254 dev eth0  metric 100 
10.1.7.0/24 dev eth0  proto kernel  scope link  src 10.1.7.1 
192.168.1.0/24 dev eth1  proto kernel  scope link  src 192.168.1.2

Do I have to change this? If so how?