SQL Performance issue due to different Execution Plan with new data

I’m hoping you all can help us with a strange issue that’s popped up suddenly after ~6 months of development on our DB environment. We've been looking for a solution the past week, but haven’t found anything similar online.

The relevant table structure is:

Our fact table:

CREATETABLE [dbo].[ts_data](

      [ModelRun_ID] [int]NOTNULL,

      [Entity_ID] [int]NOTNULL,

      [Date] [datetime]NOTNULL,

      [Value] [float]NOTNULL,

 CONSTRAINT [PK_ts_data]PRIMARYKEY CLUSTERED

(

      [ModelRun_ID]ASC,

      [Entity_ID]ASC,

      [Date]ASC

)WITH(PAD_INDEX =OFF,STATISTICS_NORECOMPUTE  =OFF, IGNORE_DUP_KEY= OFF,ALLOW_ROW_LOCKS =ON,ALLOW_PAGE_LOCKS  =ON) ON [PRIMARY]

)ON [PRIMARY]

Dimension table:

CREATETABLE [dbo].[ts_entity_def](

      [entity_id] [int]IDENTITY(1,1)NOTNULL,

      [category_id] [int]NOTNULL,

      [category] [nvarchar](50)NOTNULL,

      [type] [nvarchar](50)NOTNULL,

      [name] [nvarchar](50)NOTNULL,

 CONSTRAINT [PK_ts_entity_def]PRIMARYKEY CLUSTERED

(

      [entity_id]ASC

)WITH(PAD_INDEX =OFF,STATISTICS_NORECOMPUTE  =OFF, IGNORE_DUP_KEY= OFF,ALLOW_ROW_LOCKS =ON,ALLOW_PAGE_LOCKS  =ON) ON [PRIMARY]

)ON [PRIMARY]

The Unit table, provides more info on the dimension of interest

CREATETABLE [dbo].[unit](

      [modelrun_id] [int]NOTNULL,

      [unit_id] [int]NOTNULL,

      [unit_name] [nvarchar](64)NULL,

(... a bunch of other irrelevant fields)

)ON [PRIMARY]

We've bee using queries like the following one for ~6 months without any issues until recently when it suddenly slowed down significantly (10x):

SELECT ts.Entity_ID,date, datepart(hh,Date)as'Hour',  ValueFROM [ts_data] ts 

innerjoin ts_entity_def tsdef on ts.Entity_ID = tsdef.entity_id 

innerjoin unit u on u.unit_id= tsdef.category_idand u.modelrun_id= ts.ModelRun_ID

WHERE ts.ModelRun_ID= 1006 and tsdef.category='unit outage' and u.unit_id < 100

Each day, as we run our models, we add new data to these tables, which has not been a problem until a few days ago. All of a sudden, when we run the above query on NEW DATA it performs about 10x slower. The query above returns 672 records.

We've found two work arounds, but they are not adequate as a solution for our needs:

1.  If we re-run the statistics on these tables after the new data we want to query has been inserted
2.  If we use an OPTION (OPTIMIZEFOR (@ModelRun_ID= 1))

We've tried the following without any success:

1. Changing our indices on the various tables affected
2. Partioning the fact table (ts_data)
3. Purging the data in the fact table (was ~120 million, purged it down to ~20 million)

A bit more background:

This is a development environment so there have been some changes that coincided with the issues but we've reviewed the changes, tried to back them out, and just can’t see how any of them are connected to the underlying issue. Any clues as to where to look next would be very helpful.

Thanks!