About virtual lists

Preface

At work , We may encounter the scene of rendering big data list . The first thing we think of is the ready-made list of virtual plug-ins , however , There are so many operational requirements for the product , Online virtual list plug-ins sometimes can only do nothing ！. This article first introduces the general virtual list （ Component internal scrolling ） How to implement , Then on this basis, it expounds the idea of realizing virtual list by listening to the scrolling of external elements of components .

principle

Render some data , When scrolling, calculate what data needs to be rendered in real time .

problem

1. Since only part of the data is rendered , So how do we open the scroll bar ？

2. How the rendered data should be laid out ？

3. At what point in time to calculate what data needs to be rendered ？

Generate a simple list

First we need to have a div, It's our container element , We set a fixed height for it height, This height is transmitted from the outside of the component , When the content height is higher than height when , The container needs to be rolled , So you need to set up some css style ,overflow:auto.

<div
  className="virtualList"
  style={{
    height: `${height}px`,
  }}
  onScroll={onListScroll}
  ref={scrollRef as LegacyRef<HTMLDivElement>}
>
</div>

// css part 
.virtualList{
  position: relative;
  width: 100%;
  overflow: auto;
}
 
 Copy code 

When we have such a container , We also need to add some scrolling events to it ,onScroll={onListScroll},

const onListScroll=(e: UIEvent<HTMLDivElement>) => {
  const target = e.target as any;
  const { scrollTop } = target;
  
  console.log(scrollTop);
}
 Copy code 

So we can get the height of the scrolling content , up to now , We didn't do anything about scrolling Events .

Next , Let's continue to add something to the container element

<div
  className="virtualList"
  style={{
    height: `${height}px`,
  }}
  onScroll={onListScroll}
  ref={scrollRef as LegacyRef<HTMLDivElement>}
>
    <div className="listContent" ref={listContentRef as LegacyRef<HTMLDivElement>} > {sourceData.map((item, index) => { return ( <div className="virtualListRow" style={{ height: `${itemHeight}px`, }} key={item.key} > {renderItem(item, index)} </div> ); })} </div> </div>
 Copy code 

We render this part of the code , Very routine operation , Render all the data ,itemHeight From the outside , Indicates the height of each line ,renderItem Also from the outside , What the user wants to render each line , from renderItem decision , Here we are , We seem to have done something , It's like I didn't do anything （ Because the most important part is handed over to the user ）.

Virtual list

1. Spread the rolling height

We know that virtual lists render only part of the data , So how to open the scroll bar , Make it look as high as when rendering all the data ？ At this time, we need an element to support the height

<div
  className="virtualList"
  style={{
    height: `${height}px`,
  }}
  onScroll={onListScroll}
  ref={scrollRef as LegacyRef<HTMLDivElement>}
>
    <div className="listContent" ref={listContentRef as LegacyRef<HTMLDivElement>} > {sourceData.map((item, index) => { return ( <div className="virtualListRow" style={{ height: `${itemHeight}px`, }} key={item.key} > {renderItem(item, index)} </div> ); })} </div> {/*  Space occupying elements  */} <div style={{ height: totalHeight }}/> </div>
 Copy code 

totalHeight How to ？sourceData.length * itemHeight, At this time listContent Set to absolute positioning .

.virtualList{
  position: relative;
  width: 100%;
  overflow: auto;
  .listContent{
    position: absolute;
    width: 100%;
    z-index: 9;
  }
  .virtualListRow{
    position: relative;
  }
}
 Copy code 

2. Render some data

Next , We need to calculate what data needs to be rendered on the page , Let's first look at what data needs to be rendered during initialization .

const [position, setPosition] = useState<IPosition>({ start: 0, end: 0 });

/**  Get the amount of data that needs to be rendered on the page  */
const getRenderCount = useCallback(
(data: any[]) => {
  if (height) {
    return Math.ceil(height / itemHeight);
  }
  //  Render all 
  return sourceData.length;
},
[height, itemHeight, sourceData.length],
);

/**  Generally speaking, after the page is rendered ,renderCount Is constant , To avoid recalculating , Save this value  */
const renderCountRef = useRef<number>(0);

//  initialization   Calculation position
useEffect(() => {
const renderCount = getRenderCount(sourceDataRef.current);
renderCountRef.current = renderCount;
setPosition({ start: 0, end: renderCount });
}, [getRenderCount]);

 Copy code 

Use area height height Divided by the height of each line itemHeight, Get the amount of data you need to render when you first load , We use position This state is used to record the initial position and end position of rendering .

/**  Get the data finally rendered to the page  */
const renderData = useMemo(() => {
return sourceData.slice(position.start, position.end);
}, [position, sourceData]);
 Copy code 

renderData Is the data we want to render to the page . Then the code becomes like this ：

<div
  className="virtualList"
  style={{
    height: `${height}px`,
  }}
  onScroll={onListScroll}
  ref={scrollRef as LegacyRef<HTMLDivElement>}
>
    <div className="listContent" ref={listContentRef as LegacyRef<HTMLDivElement>} > {renderData.map((item, index) => { return ( <div className="virtualListRow" style={{ height: `${itemHeight}px`, }} key={item.key} > {renderItem(item, position.start + index)} </div> ); })} </div> {/*  Space occupying elements  */} <div style={{ height: totalHeight }}/> </div>
 Copy code 

Be careful ：renderItem(item, index) Turned into ,renderItem(item, position.start + index), Because we're going to pass it on to renderItem It is always this row of data in sourceData Index in , Not in renderData Middle index .

3. Handle scrolling logic

The above code realizes the rendering of some data , But after the scroll bar scrolls , When the data rolls out of the visual area, it's gone , therefore , Next we need to deal with the logic of scrolling

const onListScroll=(e: UIEvent<HTMLDivElement>) => {
  const target = e.target as any;
  const { scrollTop } = target;
  
  /**  Number of list items obscured  */
  const overflowNum = computedOverflowCount(sourceData, scrollTop, itemHeight);
  /**  When scrolling to the next list item , Reset the start of the rendering 、 End list item location  */
  if (overflowNum !== position.start && listContentRef.current) {
    const diff = overflowNum;
    //  Within the offset quantity, there is no need to set transform, In addition to the offset quantity, you need to set 
    if (diff > 0) {
      setPosition({
        start: diff,
        end: diff + renderCountRef.current,
      });
      //  adopt transform attribute , Reset the position to the original position , Create the illusion of rolling up all the time 
      listContentRef.current.style.transform = `translate3d(0,${computedTranslate( sourceData, diff, itemHeight, )}px,0)`;
    } else {
      setPosition({
        start: 0,
        end: overflowNum + renderCountRef.current,
      });
      listContentRef.current.style.transform = `translate3d(0,0,0)`;
    }
  }
}
 Copy code 

First, we need to calculate the rolling height scrollTop when , How many lines are covered .

/** *  Calculate the number of rows that are obscured ( Dissatisfied with one , Do not count in ) * @param overflowHeight  The height of the covered area  * @param itemHeight */
const computedOverflowCount = ( overflowHeight: number, itemHeight: IItemHeight, ) => {
  return Math.floor(overflowHeight / itemHeight);
};
 Copy code 

When you roll to the next line , Reset the start of the rendering 、 End line position , Covered by a few lines , The starting position is a few , End position is Number of lines covered + The number of lines that the visual area can render .

  setPosition({
    start: diff,
    end: diff + renderCountRef.current,
  });
  //  adopt transform attribute , Reset the position to the original position , Create the illusion of rolling up all the time 
  listContentRef.current.style.transform = `translate3d(0,${computedTranslate( sourceData, diff, itemHeight, )}px,0)`;
 Copy code 

In order to keep the data we render in the visual area all the time , We need to pass transform Attribute to put listContent Move the element down , Number of lines covered * Row height .

/** *  Calculation Translate Distance traveled  * @param count  How many  * @param itemHeight  Row height  * @returns */
const computedTranslate = ( count: number, itemHeight: IItemHeight, ) => {
  return count * itemHeight;
};
 Copy code 

such , We have completed a basic virtual list .

Dynamic row height

The height of the upper row itemHeight It's a fixed value , But in many cases, the height of each row may not be fixed , At this time, we allow itemHeight It's a function , To dynamically calculate the row height

itemHeight:number | ((rowData: any) => number);
 Copy code 

All areas involving row height need to be judged as number Type or function, Such as ：

/** *  Calculate the row height of each row  * @param rowData  Row data  * @param itemHeight  Row height  * @returns */
const computedRowHeight = (rowData: any, itemHeight: IItemHeight) => {
  if (typeof itemHeight === 'function') {
    return itemHeight(rowData);
  }
  return itemHeight;
};

/** *  Calculate the number of rows that are obscured ( Dissatisfied with one , Do not count in ) * @param sourceData  Source data  * @param overflowHeight  The height of the covered area  * @param itemHeight  Row height  */
const computedOverflowCount = ( sourceData: any[], overflowHeight: number, itemHeight: IItemHeight, ) => {
  let count = 0;
  let height = 0;
  if (typeof itemHeight === 'function') {
    sourceData.every((item) => {
      if (height <= overflowHeight) {
        height += itemHeight(item);
        count += 1;
        return true;
      }
      return false;
    });
    return count;
  }
  return Math.floor(overflowHeight / itemHeight);
};
 Copy code 

pre-render

When scrolling is fast , There will be a brief blank , To optimize this problem , We can pre render some data outside the visual area ,.

interface IVirtualList {
  ... Omit ...
  /**  Quantity offset  */
  offsetCount?: {
    before: number;
    after: number;
  };
}

... Omit ...
//  initialization   Calculation position, Need to render part of the data , namely offsetCount.after Set the number of data rows 
  useEffect(() => {
    const renderCount = getRenderCount(sourceDataRef.current);
    renderCountRef.current = renderCount;
    setPosition({ start: 0, end: renderCount + offsetCount.after });
  }, [getRenderCount, offsetCount.after]);

const onListScroll=(e: UIEvent<HTMLDivElement>) => {
  ... Omit ...
  if (overflowNum !== position.start && listContentRef.current) {
    //  You need to subtract the pre rendered part of the head , Because pre rendered lines don't have to be destroyed , So set position Count the number of pre renderings when 
    const diff = overflowNum - offsetCount.before;
    if (diff > 0) {
      setPosition({
            start: diff,
            end: overflowNum + renderCountRef.current + offsetCount.after,
          });
      //  After the pre rendered ones are rolled up ,listContent Don't move your position 
      listContentRef.current.style.transform = `translate3d(0,${computedTranslate( sourceData, offsetCount.before, diff, itemHeight, )}px,0)`;
    } else {
      ... Omit ...
    }
  }
}
 Copy code 

External scroll

At present, the virtual list components on the market are components Inside With scroll bar , The virtual list is realized by monitoring the changes of the internal scroll bar .

Now there is a requirement for components external An element has a scroll bar , When scrolling the outer scroll bar , Lists can present virtual effects .

Realization principle ： Whether it's scrolling inside the component or scrolling outside the component , The implementation principle is the same , It's just that when the component scrolls outside , The user needs to transfer the external scrolling elements to the inside of the component , So that the component can listen to the rolling event of the element .

// stay interface Join in scrollTarget Field 
interface IVirtualList {
  ... Omit ...
  /**  Height   In case of component internal scrolling, this value is required  */
  height?: number;
  /**  External scroll bar object id perhaps ref object ( Only implement first id This situation ) */
  scrollTarget?: string;
  ... Omit ...
}

/**  stay getRenderCount Some processing needs to be done in the function , Get the number of data that can be rendered in the scrolling visual area  */
  const getRenderCount = useCallback(
    (data: any[]) => {
      //  Scroll outside the component 
      if (scrollTarget) {
        const scrollDom = document.getElementById(scrollTarget);
        const scrollDomHeight = scrollDom?.offsetHeight!;
        //  If itemHeight by function, Need to dynamically calculate the renderable height 
        if (typeof itemHeight === 'function') {
          return computedRenderCount(data, scrollDomHeight, itemHeight);
        }

        return Math.ceil(scrollDomHeight / itemHeight);
      }
      //  Component internal scrolling 
      if (typeof height === 'number') {
        ... Omit ...
      }
      //  Render all 
      return sourceData.length;
    },
    [height, itemHeight, sourceData.length, scrollTarget],
  );
 Copy code 

Add listening events for external scrolling

/**  Bind listening event  */
  useEffect(() => {
    let scrollDom: HTMLElement | null = null;
    const listenerHandler = () => onOuterScroll(scrollDom);
    if (scrollTarget) {
      /**  Get the rolling element object  */
      scrollDom = document.getElementById(scrollTarget);
      scrollDom?.addEventListener('scroll', listenerHandler);
    }
    return () => {
      if (scrollDom) {
        scrollDom.removeEventListener('scroll', listenerHandler);
      }
    };
  }, [onOuterScroll, scrollTarget]);
 Copy code 

onOuterScroll The code of the function is as follows ：

const onOuterScroll=(scrollDom: HTMLElement | null)=>{
    if (scrollDom) {
        //  Gets the distance between the inner container element and the target scroll element 
        const distance = getDistance(scrollRef.current!, scrollDom);
        //  initial position 
        const initPosition = { start: 0, end: renderCountRef.current + offsetCount.after };
        if (distance < 0) {
          const absDistance = Math.abs(distance);
          const overflowNum = computedOverflowCount(sourceData, absDistance, itemHeight);
          //  When scrolling to the next list item ,
          if (overflowNum !== position.start && listContentRef.current) {
            const diff = overflowNum - offsetCount.before;
            //  Within the offset quantity, there is no need to set transform, There is no need to reset the start of rendering 、 End list item location 
            if (diff > 0) {
              setPosition({
                start: diff,
                end: overflowNum + initPosition.end,
              });

              //  adopt transform attribute , Reset the position to the original position , Create the illusion of rolling up all the time 
              listContentRef.current.style.transform = `translate3d(0,${computedTranslate( sourceData, offsetCount.before, diff, itemHeight, )}px,0)`;
            } else {
              ... Omit   Code is the same as internal scrolling ...
            }
          }
        }
        //  If there is no return to the initial value , It needs to be set to the initial value 
        else if (position.start !== initPosition.start || position.end !== initPosition.end) {
          setPosition({ ...initPosition });
          if (listContentRef.current) {
            listContentRef.current.style.transform = `translate3d(0,0,0)`;
          }
        }
      }
}
 Copy code 

Be careful ： The difference between external scrolling and internal scrolling is , Inside, we scroll through scrollTop Directly calculate the hidden row tree overflowNum, But when the outside scrolls , We need to get The distance between the inner container element and the target scroll element To calculate overflowNum,distance It could be a positive number , It could also be negative .

Pictured ：

• Internal scrolling

• External scroll ,distance In the case of positive numbers

• External scroll ,distance negative

such , We implemented a monitor for external scrolling , Realize the function of virtual list .

The effect is as follows ：

We can see on the right , When scrolling, some elements do not change . New elements change , The rest of the elements remain the same , Avoid unnecessary rendering .